{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font size=6>支持向量机</font>\n",
    "# SVM\n",
    "SVM的全称是Support Vector Machine，即支持向量机，主要用于解决模式识别领域中的数据分类问题，属于有监督学习算法的一种。支持向量机虽然诞生只有短短的二十多年，但是自一诞生便由于它良好的分类性能席卷了机器学习领域，并牢牢压制了神经网络领域好多年。如果不考虑集成学习的算法，不考虑特定的训练数据集，在分类算法中的表现SVM可以说是最优异的。\n",
    "\n",
    "SVM支持二元分类，线性分类和非线性分类。经过演进，现在也可以支持多元分类，同时经过扩展，也能应用于回归问题。\n",
    "\n",
    "# 线性支持向量机\n",
    "## 回顾感知机模型\n",
    "1. 标签$y$值为$-1或1$\n",
    "2. 分离超平面：$w^Tx + b = 0$\n",
    "3. 感知机损失函数：$\\sum\\limits_{x_i \\in M}- y^{(i)}(w^Tx^{(i)} +b)\\big / ||w||_2\\to\\sum\\limits_{x_i \\in M}- y^{(i)}(w^Tx^{(i)} +b)$\n",
    "<img src=\"images/1042406-20161124135616081-623185925.jpg\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 函数间隔与几何间隔\n",
    "在感知机模型中，对于分离超平面$w^Tx + b = 0$，$|w^Tx + b |$表示点$x$到超平面的相对距离。通过观察$w^Tx + b$和$y$是否同号，可以判断分类是否正确。这里函数间隔定义为$\\gamma^{'} = y(w^Tx + b)$。\n",
    "\n",
    "函数间隔并不能正常反应点到超平面的距离，几何间隔定义为$\\gamma = \\frac{y(w^Tx + b)}{||w||_2}$，几何间隔才是点到超平面的真正距离，感知机模型里用到的距离就是几何距离。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 支持向量\n",
    "<img width=\"60%\" src=\"images/1042406-20161124144326487-1331861308.jpg\">\n",
    "\n",
    "**总结：**\n",
    "1. 感知机目标是是所有点尽可能的原理分离超平面；\n",
    "2. 支持向量机，不考虑较远的点，只考虑距离超平面较近的点，使其尽可能距离超平面较远，这些点叫做支持向量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SVM模型目标函数与优化\n",
    "SVM的模型是让所有点到超平面的距离大于一定的距离，也就是所有的分类点要在各自类别的支持向量两边。用数学式子表示为：\n",
    "$$max \\;\\; \\gamma = \\frac{y(w^Tx + b)}{||w||_2}  \\;\\; s.t \\;\\; y_i(w^Tx_i + b) = \\gamma^{'(i)} \\geq \\gamma^{'} (i =1,2,...m)$$\n",
    "一般我们都取函数间隔$\\gamma^{'} = y(w^Tx + b)=1$，这样我们的优化函数定义为：\n",
    "$$max \\;\\; \\frac{1}{||w||_2}  \\;\\; s.t \\;\\; y_i(w^Tx_i + b)  \\geq 1 (i =1,2,...m)$$\n",
    "也就是说，我们要在约束条件$y_i(w^Tx_i + b)  \\geq 1 (i =1,2,...m)$下，最大化$\\frac{1)}{||w||_2}$。**可以看出，这里的优化方式不同，感知机是固定分母优化分子，而SVM是固定分子优化分母，同时加上了支持向量的限制。**\n",
    "\n",
    "由于$\\frac{1}{||w||_2}$的最大化等同于$\\frac{1}{2}||w||_2^2$的最小化。这样SVM的优化函数等价于\n",
    "$$min \\;\\; \\frac{1}{2}||w||_2^2  \\;\\; s.t \\;\\; y_i(w^Tx_i + b)  \\geq 1 (i =1,2,...m)$$\n",
    "由于目标函数$\\frac{1}{2}||w||_2^2$是凸函数，同时约束条件不等式是仿射的，根据凸优化理论，我们可以通过拉格朗日函数将我们的优化目标转化为无约束的优化函数\n",
    "$$L(w,b,\\alpha) = \\frac{1}{2}||w||_2^2 - \\sum\\limits_{i=1}^{m}\\alpha_i[y_i(w^Tx_i + b) - 1] \\; 满足\\alpha_i \\geq 0$$\n",
    "其中$\\alpha$为拉格朗日乘子向量。\n",
    "\n",
    "由于引入了拉格朗日乘子，我们的优化目标变成：\n",
    "$$\\underbrace{min}_{w,b}\\; \\underbrace{max}_{\\alpha_i \\geq 0} L(w,b,\\alpha)$$\n",
    "我们的这个优化函数满足KKT条件，也就是说，我们可以通过拉格朗日对偶将我们的优化问题转化为等价的对偶问题来求解。也就是说，现在我们要求的是：\n",
    "$$\\underbrace{max}_{\\alpha_i \\geq 0} \\;\\underbrace{min}_{w,b}\\;  L(w,b,\\alpha)$$\n",
    "\n",
    "注：详细拉格朗日函数以及对偶化推导过程见:**[零基础学SVM](https://zhuanlan.zhihu.com/p/24638007)和[如何理解拉格朗日乘子法和KKT条件？](https://www.matongxue.com/madocs/987.html)**\n",
    "\n",
    "从上式中，我们可以先求优化函数对于$w和b$的极小值。接着再求拉格朗日乘子$α$的极大值。\n",
    "\n",
    "首先，我们来求$L(w,b,\\alpha) $基于$w和b$的极小值，即$\\underbrace{min}_{w,b}\\;  L(w,b,\\alpha)$。这个极值我们可以通过对$w和b$分别求偏导数得到：\n",
    "$$\\frac{\\partial L}{\\partial w} = 0 \\;\\Rightarrow w = \\sum\\limits_{i=1}^{m}\\alpha_iy_ix_i\\\\\\frac{\\partial L}{\\partial b} = 0 \\;\\Rightarrow \\sum\\limits_{i=1}^{m}\\alpha_iy_i = 0$$\n",
    "从上两式子可以看出，我们已经求得了$w和α$的关系，只要我们后面接着能够求出优化函数极大化对应的$α$，就可以求出我们的$w$了，至于$b$，由于上两式已经没有$b$，所以最后的$b$可以有多个。\n",
    "\n",
    "定义：\n",
    "$$\\psi(\\alpha) = \\underbrace{min}_{w,b}\\;  L(w,b,\\alpha)$$\n",
    "\n",
    "将$w$替换为$\\alpha$的表达式以后的优化函数(**省略详细推导过程**)：\n",
    "$$\\begin{align}\\psi(\\alpha) \n",
    "& =\\underbrace{min}_{w,b}\\;  L(w,b,\\alpha)\\\\ \n",
    "& =\\underbrace{min}_{w,b}\\;  \\frac{1}{2}||w||_2^2 - \\sum\\limits_{i=1}^{m}\\alpha_i[y_i(w^Tx_i + b) - 1] \\\\\n",
    "& =\\underbrace{min}_{w,b}\\;  \\sum\\limits_{i=1}^{m}\\alpha_i  - \\frac{1}{2}\\sum\\limits_{i=1,j=1}^{m}\\alpha_i\\alpha_jy_iy_jx_i^Tx_j\\\\\n",
    "& =\\sum\\limits_{i=1}^{m}\\alpha_i  - \\frac{1}{2}\\sum\\limits_{i=1,j=1}^{m}\\alpha_i\\alpha_jy_iy_jx_i^Tx_j\n",
    "\\end{align}$$\n",
    "\n",
    "从上面可以看出，通过对$w,b$极小化后，我们的优化函数$\\psi (\\alpha)$仅仅只有$\\alpha$向量做参数，消除了$w,b$参数。接下来我们极大化$\\psi (\\alpha)$，就可以求出此时对应的$\\alpha$，进而求出$w,b$。\n",
    "\n",
    "$$$$\n",
    "$$\\begin{align}\n",
    "& \\underbrace{max}_{\\alpha_i \\geq 0} \\;\\underbrace{min}_{w,b}\\;L(w,b,\\alpha)\\\\\n",
    "&= \\underbrace{max}_{\\alpha_i \\geq 0} \\;\\psi(\\alpha)\\\\\n",
    "&= \\underbrace{max}_{\\alpha} -\\frac{1}{2}\\sum\\limits_{i=1}^{m}\\sum\\limits_{j=1}^{m}\\alpha_i\\alpha_jy_iy_j(x_i \\bullet x_j) + \\sum\\limits_{i=1}^{m} \\alpha_i\\\\\n",
    "&= \\underbrace{min}_{\\alpha} \\frac{1}{2}\\sum\\limits_{i=1}^{m}\\sum\\limits_{j=1}^{m}\\alpha_i\\alpha_jy_iy_j(x_i \\bullet x_j) -  \\sum\\limits_{i=1}^{m} \\alpha_i\\\\\n",
    "&s.t. \\; \\sum\\limits_{i=1}^{m}\\alpha_iy_i = 0\\\\\n",
    "&\\alpha_i \\geq 0  \\; i=1,2,...m\n",
    "\\end{align}$$\n",
    "\n",
    "\n",
    "\n",
    "只要我们可以求出上式极小化时对应的$α$向量就可以求出$w和b$了。具体怎么极小化上式得到对应的$α$，一般需要用到SMO算法，这个算法比较复杂，后面会有介绍。在这里，我们假设通过SMO算法，我们得到了对应的$α$的值$\\alpha ^*$。\n",
    "\n",
    "我们根据$w = \\sum\\limits_{i=1}^{m}\\alpha_iy_ix_i$，可以求出对应的$w$的值$w^{*} = \\sum\\limits_{i=1}^{m}\\alpha_i^{*}y_ix_i$。\n",
    "\n",
    "求$b$则稍微麻烦一点。注意到，对于任意支持向量$(x_s, y_s)$，都有$y_s(w^Tx_s+b) = y_s(\\sum\\limits_{i=1}^{m}\\alpha_iy_ix_i^Tx_s+b) = 1$。假设我们有$S$个支持向量，则对应我们求出$S个b^*$，理论上这些$b^*$都可以作为最终的结果， 但是我们一般采用一种更健壮的办法，即求出所有支持向量所对应的$b^*_s$，然后将其平均值作为最后的结果。注意到对于严格线性可分的SVM，$b$的值是有唯一解的，也就是这里求出的所有$b^*$都是一样的，这里我们仍然这么写是为了和后面加入软间隔后的SVM的算法描述一致。\n",
    "\n",
    "根据KKT条件中的对偶互补条件$\\alpha_{i}^{*}(y_i(w^Tx_i + b) - 1) = 0$(**详见推荐文章**)，如果$\\alpha_i>0$则有$y_i(w^Tx_i + b) =1$即点为支持向量，否则如果$\\alpha_i=0$则有$y_i(w^Tx_i + b)\\geq 1$，即样本为支持向量或者已经被正确分类。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 线性可分SVM的算法过程\n",
    "**输入：**\n",
    "\n",
    "线性可分的$m$个样本$(x_1,y_1),\\dots,(x_m,y_m)$，其中$x$为$n$维特征向量。$y$为二元输出，值为$-1或1$。\n",
    "\n",
    "**输出：**\n",
    "\n",
    "分离超平面的参数$w^*和b^*$和分类决策函数。\n",
    "\n",
    "**算法过程：**\n",
    "1. 构造约束优化问题：$$\\underbrace{min}_{\\alpha} \\frac{1}{2}\\sum\\limits_{i=1}^{m}\\sum\\limits_{j=1}^{m}\\alpha_i\\alpha_jy_iy_j(x_i \\bullet x_j) -  \\sum\\limits_{i=1}^{m} \\alpha_i\\\\s.t. \\; \\sum\\limits_{i=1}^{m}\\alpha_iy_i = 0\\\\\\alpha_i \\geq 0  \\; i=1,2,...m$$\n",
    "2. 用SMO算法求出上式最小时对应的$\\alpha$向量的值$\\alpha^*$向量。\n",
    "3. 计算$w^{*} = \\sum\\limits_{i=1}^{m}\\alpha_i^{*}y_ix_i$\n",
    "4. 找出所有的$S$个支持向量，即满足$\\alpha_s>0$对应的样本$(x_s,y_s)$，通过$y_s(\\sum\\limits_{i=1}^{m}\\alpha_iy_ix_i^Tx_s+b) = 1$，计算出每个支持向量$(x_s,y_s)$对应的$b_s^*$，计算公式为$b_s^{*} = y_s - \\sum\\limits_{i=1}^{s}\\alpha_iy_ix_i^Tx_s$。所有的$b_s^*$对应的平均值即为最终的$b^{*} = \\frac{1}{S}\\sum\\limits_{i=1}^{S}b_s^{*}$。\n",
    "5. 最终的分类超平面为$w^{*} \\bullet x + b^{*} = 0$，最终的分类决策函数为$f(x) = sign(w^{*} \\bullet x + b^{*})$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 线性支持向量机的软间隔最大化模型\n",
    "## 线性分类SVM面临的问题\n",
    "由于异常点，导致不能线性分类或者弱化分类效果\n",
    "<table style=\"width:80%\"><tr>\n",
    "    <td>不能线性分类</td><td>分类效果不佳</td>\n",
    "    </tr><tr>\n",
    "    <td><img src=\"images/1042406-20161125104106409-1177897648.png\"></td>\n",
    "    <td><img src=\"images/1042406-20161125104737206-364720074.png\"></td>\n",
    "    </tr></table>\n",
    "\n",
    "## 线性分类SVM的软间隔最大化\n",
    "软间隔，是相对于硬间隔说的，上述线性分类SVM的学习方法属于硬间隔最大化。\n",
    "\n",
    "已知硬间隔最大化的条件：\n",
    "$$min \\;\\; \\frac{1}{2}||w||_2^2  \\;\\; s.t \\;\\; y_i(w^Tx_i + b)  \\geq 1 (i =1,2,...m)$$\n",
    "\n",
    "软间隔最大化，SVM对训练集里面的每个样本$(x_i,y_i)$引入了一个松弛变量$\\xi_i \\geq 0$，使函数间隔加上松弛变量大于等于1，也就是说：\n",
    "$$y_i(w\\bullet x_i +b) \\geq 1- \\xi_i$$\n",
    "<img src=\"images/784325-20190106205611985-1231595363.png\">\n",
    "**如图：**\n",
    "1. 在硬间隔中：决策平面是绿色实线，约束条件是要保证两类样本点分别位于两条绿色虚线的两侧，所有样本点都被正确分类。\n",
    "2. 在软间隔中：决策平面是绿色实线，约束条件变为蓝色虚线，且红色样本点在红色箭头一侧即可，紫色样本点在紫色箭头一侧即可。\n",
    "\n",
    "对比硬间隔最大化，可以看到我们对样本到超平面的函数距离的要求放松了，之前是一定要大于等于1，现在只需要加上一个大于等于0的松弛变量能大于等于1就可以了。当然，松弛变量不能白加，这是有成本的，每一个松弛变量$\\xi_i$，对应了一个代价$\\xi_i$，从而我们的软间隔最大化的SVM学习条件如下：\n",
    "$$min\\;\\; \\frac{1}{2}||w||_2^2 +C\\sum\\limits_{i=1}^{m}\\xi_i\\\\s.t.  \\;\\; y_i(w^Tx_i + b)  \\geq 1 - \\xi_i \\;\\;(i =1,2,...m)\\\\\\xi_i \\geq 0 \\;\\;(i =1,2,...m)$$\n",
    "\n",
    "这里,$C>0$为惩罚参数，可以理解为我们一般回归和分类问题正则化时候的参数。$C$越大，对误分类的惩罚越大，$C$越小，对误分类的惩罚越小。也就是说，我们希望$\\frac{1}{2}||w||_2^2 $尽量小，误分类的点尽可能的少。$C$是协调两者关系的正则化惩罚系数。在实际应用中，需要调参来选择。\n",
    "## 线性分类SVM的软间隔最大化目标函数的优化\n",
    "线性可分SVM的优化方式类似，我们首先将软间隔最大化的约束问题用拉格朗日函数转化为无约束问题如下：\n",
    "$$L(w,b,\\xi,\\alpha,\\mu) = \\frac{1}{2}||w||_2^2 +C\\sum\\limits_{i=1}^{m}\\xi_i - \\sum\\limits_{i=1}^{m}\\alpha_i[y_i(w^Tx_i + b) - 1 + \\xi_i] - \\sum\\limits_{i=1}^{m}\\mu_i\\xi_i$$\n",
    "其中$\\mu_i \\geq 0, \\alpha_i \\geq 0$，均为拉格朗日系数。\n",
    "\n",
    "也就是说，我们现在要优化的目标函数是：$$\\underbrace{min}_{w,b,\\xi}\\; \\underbrace{max}_{\\alpha_i \\geq 0, \\mu_i \\geq 0,} L(w,b,\\alpha, \\xi,\\mu)$$\n",
    "\n",
    "这个优化目标也满足KKT条件，也就是说，我们可以通过拉格朗日对偶将我们的优化问题转化为等价的对偶问题来求解如下：$$\\underbrace{max}_{\\alpha_i \\geq 0, \\mu_i \\geq 0,} \\; \\underbrace{min}_{w,b,\\xi}\\; L(w,b,\\alpha, \\xi,\\mu)$$\n",
    "\n",
    "我们可以先求优化函数对于$w, b, \\xi$的极小值, 接着再求拉格朗日乘子$\\alpha$和 $\\mu$的极大值。\n",
    "\n",
    "首先我们来求优化函数对于$w, b, \\xi$的极小值，这个可以通过求偏导数求得：\n",
    "$$\\frac{\\partial L}{\\partial w} = 0 \\;\\Rightarrow w = \\sum\\limits_{i=1}^{m}\\alpha_iy_ix_i\\\\\\frac{\\partial L}{\\partial b} = 0 \\;\\Rightarrow \\sum\\limits_{i=1}^{m}\\alpha_iy_i = 0\\\\\\frac{\\partial L}{\\partial \\xi} = 0 \\;\\Rightarrow C- \\alpha_i - \\mu_i = 0$$\n",
    "\n",
    "我们可以利用上面的三个式子去消除$w, b$。\n",
    "$$\\begin{align} L(w,b,\\xi,\\alpha,\\mu) \n",
    "& = \\frac{1}{2}||w||_2^2 +C\\sum\\limits_{i=1}^{m}\\xi_i - \\sum\\limits_{i=1}^{m}\\alpha_i[y_i(w^Tx_i + b) - 1 + \\xi_i] - \\sum\\limits_{i=1}^{m}\\mu_i\\xi_i 　\\\\\n",
    "& = \\sum\\limits_{i=1}^{m}\\alpha_i - \\frac{1}{2}\\sum\\limits_{i=1,j=1}^{m}\\alpha_i\\alpha_jy_iy_jx_i^Tx_j \n",
    "\\end{align}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上式，和我们硬间隔SVM拉格朗日函数推导式一样，唯一不一样的是约束条件。现在我们的优化目标的数学形式为：\n",
    "$$\\begin{align}\n",
    "&\\underbrace{max}_{\\alpha_i \\geq 0, \\mu_i \\geq 0,} \\; \\underbrace{min}_{w,b,\\xi}\\; L(w,b,\\alpha, \\xi,\\mu)\\\\\n",
    "&=\\underbrace{ max }_{\\alpha} \\sum\\limits_{i=1}^{m}\\alpha_i - \\frac{1}{2}\\sum\\limits_{i=1,j=1}^{m}\\alpha_i\\alpha_jy_iy_jx_i^Tx_j\\\\\n",
    "&=\\underbrace{min}_{\\alpha} \\frac{1}{2}\\sum\\limits_{i=1}^{m}\\sum\\limits_{j=1}^{m}\\alpha_i\\alpha_jy_iy_j(x_i \\bullet x_j) -  \\sum\\limits_{i=1}^{m} \\alpha_i\\\\\n",
    "&s.t. \\; \\sum\\limits_{i=1}^{m}\\alpha_iy_i = 0\\\\\n",
    "&{\\color {red}C}\\geq \\alpha_i \\geq 0  \\; i=1,2,...m\\\\\n",
    "\\end{align}$$\n",
    "\n",
    "这就是软间隔最大化时的线性可分SVM的优化目标形式，和硬间隔最大化的线性可分SVM相比，我们多了一个约束条件${\\color {red}C}\\geq \\alpha_i \\geq 0$。\n",
    "\n",
    "这样，我们依然可以通过SMO算法来求上式极小化时对应的$α$向量就可以求出$w和b$了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 软间隔最大化时的支持向量\n",
    "在硬间隔最大化时，支持向量比较简单。在软间隔最大化时，则稍微复杂一些，因为我们对每个样本$(x_i,y_i)$引入了松弛变量$\\xi_i$。根据KKT条件中的对偶互补条件$\\alpha_{i}^{*}(y_i(w^Tx_i + b) - 1+\\xi_i) = 0$(**详见推荐文章**)。可知，$\\alpha_i>0$(即$\\alpha_i\\not =0$)的样本为支持向量，在间隔边界上、间隔边界与决策超平面之间、或者在超平面误分一侧的向量都有可能是支持向量，这是因为软间隔模型中的每个样本的$\\alpha_i$、$\\xi_i$不同。\n",
    "\n",
    "我们从下图来研究软间隔最大化是支持向量的情况，第$i$个点对应类别支持向量的距离为$\\frac{\\xi_i}{||w||_2}$：\n",
    "1. 如果$\\alpha_i=0$，那么$y_i(w^Tx_i + b) - 1 +\\xi_i\\geq 0$,即样本被正确分类。\n",
    "2. 如果$0 < \\alpha_i < C$，那么根据你KKT条件中的互补条件$\\mu_i\\xi_i=0$,以及$C-\\alpha_i-\\mu_i=0$，可得$\\xi_i=0$。另由于$y_i(w^Tx_i + b) - 1 +\\xi_i=0$，则$y_i(w^Tx_i + b) = 1$，即该点为支持向量。\n",
    "3. 如果$\\alpha=C$，说明这是一个可能比较异常的点，需要检查此时$\\xi_i$\n",
    "    1. 如果$0\\leq\\xi_i<1$，那么点被正确分类，但是却在超平面和自己类别的间隔边界之间。如图中的样本2和4。\n",
    "    2. 如果$\\xi_i=1$，那么点在分离超平面上，无法被正确分类。\n",
    "    3. 如果$\\xi_i>1$，那么点在超平面的另一侧，也就是说，这个点不能被正常分类。如图中的样本1和3。\n",
    "    \n",
    "<img width=\"50%\" src=\"images/1042406-20161125133202346-307657619.jpg\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 线性不可分支持向量机与核函数\n",
    "上面我们讲到了线性可分SVM的硬间隔最大化和软间隔最大化的算法，它们对线性可分的数据有很好的处理，但是对完全线性不可分的数据没有办法。接下来，我们来探讨SVM如何处理线性不可分的数据，重点讲述核函数在SVM中处理线性不可分数据的作用。\n",
    "## 回顾多项式回归\n",
    "之前在线性回归原理中，讲过可以将多项式回归转化为线性回归。如一个只有两个特征的$p$次方多项式回归的模型：\n",
    "$$h_\\theta(x_1, x_2) = \\theta_0 + \\theta_{1}x_1 + \\theta_{2}x_{2} + \\theta_{3}x_1^{2} + \\theta_{4}x_2^{2} + \\theta_{5}x_{1}x_2$$\n",
    "\n",
    "我们令$x_0=1,x_1=x_1,x_2=x_2,x_3=x_1^2,x_4=x_2^2,x_5=x_1x_2$，这样我们就得到了下式：\n",
    "$$h_\\theta(x_1, x_2) = \\theta_0 + \\theta_{1}x_1 + \\theta_{2}x_{2} + \\theta_{3}x_3 + \\theta_{4}x_4 + \\theta_{5}x_5$$\n",
    "\n",
    "可以发现，我们又重新回到了线性回归，这是一个五元线性回归，可以用线性回归的方法来完成算法。对于每个二元样本特征$(x_1,x_2)$,我们得到一个五元样本特征$(1, x_1, x_2, x_{1}^2, x_{2}^2, x_{1}x_2)$，通过这个改进的五元样本特征，我们重新把不是线性回归的函数变回线性回归。\n",
    "\n",
    "也就是说，对于二维的不是线性的数据，我们将其映射到了五维以后，就变成了线性的数据。\n",
    "\n",
    "总结来说，对于在低维线性不可分的数据，在映射到了高维以后，就变成线性可分的了。这个思想我们同样可以运用到SVM的线性不可分数据上。对于SVM线性不可分的低维特征数据，我们可以将其映射到高维，就能线性可分，此时就可以运用线性可分SVM的算法思想了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 核函数的引入\n",
    "线性可分SVM的优化目标函数：\n",
    "$$\\underbrace{ min }_{\\alpha}  \\frac{1}{2}\\sum\\limits_{i=1,j=1}^{m}\\alpha_i\\alpha_jy_iy_jx_i \\bullet x_j - \\sum\\limits_{i=1}^{m}\\alpha_i\\\\\n",
    "s.t. \\; \\sum\\limits_{i=1}^{m}\\alpha_iy_i = 0\\\\\n",
    "0 \\leq \\alpha_i \\leq C\n",
    "$$\n",
    "\n",
    "上式低维特征仅仅以内积$$x_i \\bullet x_j$$的形式出现，如果我们定义一个低维特征空间到高维特征空间的映射$\\phi$(例如，上面2维到5维的映射)，将所有特征映射到一个更高的维度，让数据线性可分，我们就可以继续按线性可分SVM的算法思想来优化目标函数，求出分离超平面和分类决策函数了。也就是说现在的SVM的优化目标函数变成：\n",
    "$$\\underbrace{ min }_{\\alpha}  \\frac{1}{2}\\sum\\limits_{i=1,j=1}^{m}\\alpha_i\\alpha_jy_iy_j\\phi(x_i) \\bullet \\phi(x_j) - \\sum\\limits_{i=1}^{m}\\alpha_i\\\\\n",
    "s.t. \\; \\sum\\limits_{i=1}^{m}\\alpha_iy_i = 0\\\\\n",
    "0 \\leq \\alpha_i \\leq C\n",
    "$$\n",
    "\n",
    "和线性可分SVM的优化目标函数的区别仅仅是将内积$x_i \\bullet x_j$替换为$\\phi(x_i) \\bullet \\phi(x_j)$。\n",
    "\n",
    "这时候，假如是一个2维特征的数据，我们可以将其映射到5维来做特征的内积，如果原始空间是三维，可以映射到到19维空间，似乎还可以处理。但是如果我们的低维特征是100个维度，1000个维度。那么我们要将其映射到超级高的维度来计算特征的内积。这时候映射成的高维维度是爆炸性增长的，这个计算量实在是太大了，而且如果遇到无穷维的情况，就根本无从计算。\n",
    "\n",
    "这里需要引用核函数，解决上述问题。假设$\\phi$是一个从低维的输入空间$\\chi$（欧式空间的子集或者离散集合）到高维的希尔伯特空间的$\\mathcal{H}$映射。那么如果存在函数$K(y,z)$，对于任意$y, z \\in \\chi$，都有：\n",
    "$$K(y, z) = \\phi(x_i) \\bullet \\phi(x_j)$$\n",
    "\n",
    "那么我们就称$K(y,z)$为核函数。$K(y,z)$的计算是在低维特征空间来计算的，它避免了上面提到的在高维维度空间计算内积的恐怖计算量的同时，实现了在高维特征空间线性可分的目的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 核函数的介绍\n",
    "### 线性核函数\n",
    "线性核函数（Linear Kernel）其实就是我们线性可分SVM，表达式为：\n",
    "$$K(x, z) = x \\bullet z$$\n",
    "\n",
    "也就是说，线性可分SVM我们可以和线性不可分SVM归为一类，区别仅仅在于线性可分SVM用的是线性核函数。\n",
    "\n",
    "### 多项式核函数\n",
    "多项式核函数（Polynomial Kernel）是线性不可分SVM常用的核函数之一，表达式为：\n",
    "$$K(x, z) = （\\gamma x \\bullet z  + r)^d$$\n",
    "\n",
    "其中，$\\gamma, r, d$都需要自己调参定义。\n",
    "\n",
    "### 高斯核函数\n",
    "高斯核函数（Gaussian Kernel），在SVM中也称为径向基核函数（Radial Basis Function,RBF），它是非线性分类SVM最主流的核函数。表达式为：\n",
    "$$K(x, z) = exp(-\\gamma||x-z||^2)$$\n",
    "\n",
    "其中，$\\gamma>0$，需要自己调参定义。\n",
    "### Sigmoid核函数\n",
    "Sigmoid核函数（Sigmoid Kernel）也是线性不可分SVM常用的核函数之一，表达式为：\n",
    "$$K(x, z) = tanh（\\gamma x \\bullet z  + r)$$\n",
    "\n",
    "其中，$\\gamma,r$都需要自己调参定义。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 线性支持回归\n",
    "前面讲了SVM的线性分类和非线性分类，实际上SVM也可以用于回归。\n",
    "## SVM回归模型的损失函数度量\n",
    "前面SVM分类模型中，我们的目标函数是让$\\frac{1}{2}||w||_2^2$最小，同时让各个训练集中的点尽量远离自己类别一边的的支持向量，即$y_i(w \\bullet \\phi(x_i )+ b) \\geq 1$。如果是加入一个松弛变量$\\xi_i \\geq 0$,则目标函数是$\\frac{1}{2}||w||_2^2 +C\\sum\\limits_{i=1}^{m}\\xi_i$，对应的约束条件变成：$y_i(w \\bullet \\phi(x_i ) + b )  \\geq 1 - \\xi_i$。\n",
    "\n",
    "对于回归模型，优化目标函数可以继续和SVM分类模型保持一致为$\\frac{1}{2}||w||_2^2$，对于约束条件，我们的目标是让训练集中的每一个点，尽量拟合到一个线性模型$y_i ~= w \\bullet \\phi(x_i ) +b$。对于一般的回归模型，我们是用均方差作为损失函数,但是SVM不是这样定义损失函数的。\n",
    "\n",
    "SVM需要我们定义一个常量$\\epsilon > 0$,对于某一个点$(x_i,y_i)$，如果$|y_i - w \\bullet \\phi(x_i ) -b| \\leq \\epsilon$，则完全没有损失，如果$|y_i - w \\bullet \\phi(x_i ) -b| > \\epsilon$,则对应的损失为$|y_i - w \\bullet \\phi(x_i ) -b| - \\epsilon$。\n",
    "\n",
    "如下图所示，在蓝色条带里面的点都是没有损失的，但是外面的点的是有损失的，损失大小为红色线的长度。\n",
    "<img src=\"images/1042406-20161129125204240-1430845027.png\">\n",
    "\n",
    "最终，SVM回归模型的损失函数度量为：\n",
    "$$err(x_i,y_i) =  \\begin{cases} 0 & {|y_i - w \\bullet \\phi(x_i ) -b| \\leq \\epsilon}\\\\ |y_i - w \\bullet \\phi(x_i ) -b| - \\epsilon & {|y_i - w \\bullet \\phi(x_i ) -b| > \\epsilon} \\end{cases}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SVM回归模型的目标函数的原始形式\n",
    "我们的目标函数如下：\n",
    "$$min\\;\\; \\frac{1}{2}||w||_2^2  \\;\\; s.t \\;\\; |y_i - w \\bullet \\phi(x_i ) -b| \\leq \\epsilon (i =1,2,...m)$$\n",
    "\n",
    "和SVM分类模型相似，回归模型也可以对每个样本$(x_i,y_i)$加入松弛变量$\\xi_i \\geq 0$, 但是由于我们这里用的是绝对值，实际上是两个不等式，也就是说两边都需要松弛变量，我们定义为$\\xi_i^{\\lor}, \\xi_i^{\\land}$, 则我们SVM回归模型的损失函数度量在加入松弛变量之后变为：\n",
    "$$min\\;\\; \\frac{1}{2}||w||_2^2 + C\\sum\\limits_{i=1}^{m}(\\xi_i^{\\lor}+ \\xi_i^{\\land})\\\\\n",
    "s.t. \\;\\;\\; -\\epsilon - \\xi_i^{\\lor} \\leq y_i - w \\bullet \\phi(x_i ) -b \\leq \\epsilon + \\xi_i^{\\land}\\\\\n",
    "\\xi_i^{\\lor} \\geq 0, \\;\\; \\xi_i^{\\land} \\geq 0 \\;(i = 1,2,..., m)$$\n",
    "\n",
    "依然和SVM分类模型相似，我们可以用拉格朗日函数将目标优化函数变成无约束的形式，也就是拉格朗日函数的原始形式如下：\n",
    "$$L(w,b,\\alpha^{\\lor}, \\alpha^{\\land}, \\xi_i^{\\lor}, \\xi_i^{\\land}, \\mu^{\\lor}, \\mu^{\\land}) \\\\\n",
    "= \\frac{1}{2}||w||_2^2 + C\\sum\\limits_{i=1}^{m}(\\xi_i^{\\lor}+ \\xi_i^{\\land}) + \\sum\\limits_{i=1}^{m}\\alpha^{\\lor}(-\\epsilon - \\xi_i^{\\lor} -y_i + w \\bullet \\phi(x_i) + b) + \\sum\\limits_{i=1}^{m}\\alpha^{\\land}(y_i - w \\bullet \\phi(x_i ) - b -\\epsilon - \\xi_i^{\\land}) - \\sum\\limits_{i=1}^{m}\\mu^{\\lor}\\xi_i^{\\lor} - \\sum\\limits_{i=1}^{m}\\mu^{\\land}\\xi_i^{\\land}$$\n",
    "\n",
    "其中，$\\mu^{\\lor} \\geq 0, \\mu^{\\land} \\geq 0, \\alpha_i^{\\lor} \\geq 0, \\alpha_i^{\\land} \\geq 0$，均为拉格朗日系数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SVM回归模型的目标函数的对偶形式\n",
    "上面，我们讲到了SVM回归模型的目标函数的原始形式,我们的目标是\n",
    "$$\\underbrace{min}_{w,b,\\xi_i^{\\lor}, \\xi_i^{\\land}}\\; \\;\\;\\;\\;\\;\\;\\;\\;\\underbrace{max}_{\\mu^{\\lor} \\geq 0, \\mu^{\\land} \\geq 0, \\alpha_i^{\\lor} \\geq 0, \\alpha_i^{\\land} \\geq 0}\\;L(w,b,\\alpha^{\\lor}, \\alpha^{\\land}, \\xi_i^{\\lor}, \\xi_i^{\\land}, \\mu^{\\lor}, \\mu^{\\land})$$\n",
    "\n",
    "和SVM分类模型一样，这个优化目标也满足KKT条件，也就是说，我们可以通过拉格朗日对偶将我们的优化问题转化为等价的对偶问题来求解如下：\n",
    "$$\\underbrace{max}_{\\mu^{\\lor} \\geq 0, \\mu^{\\land} \\geq 0, \\alpha_i^{\\lor} \\geq 0, \\alpha_i^{\\land} \\geq 0}\\; \\;\\;\\;\\;\\;\\;\\;\\;\\underbrace{min}_{w,b,\\xi_i^{\\lor}, \\xi_i^{\\land}}\\;L(w,b,\\alpha^{\\lor}, \\alpha^{\\land}, \\xi_i^{\\lor}, \\xi_i^{\\land}, \\mu^{\\lor}, \\mu^{\\land})$$\n",
    "\n",
    "我们先求优化函数对于$w,b,\\xi_i^{\\lor}, \\xi_i^{\\land}$的极小值, 接着再求拉格朗日乘子$\\alpha^{\\lor}, \\alpha^{\\land}, \\mu^{\\lor}, \\mu^{\\land}$的极大值。\n",
    "\n",
    "首先来求优化函数对于$w,b,\\xi_i^{\\lor}, \\xi_i^{\\land}$的极小值，这个可以通过求偏导数求得：\n",
    "$$\\frac{\\partial L}{\\partial w} = 0 \\;\\Rightarrow w = \\sum\\limits_{i=1}^{m}(\\alpha_i^{\\land} - \\alpha_i^{\\lor})\\phi(x_i)\\\\\n",
    "\\frac{\\partial L}{\\partial b} = 0 \\;\\Rightarrow  \\sum\\limits_{i=1}^{m}(\\alpha_i^{\\land} - \\alpha_i^{\\lor}) = 0\\\\\n",
    "\\frac{\\partial L}{\\partial \\xi_i^{\\lor}} = 0 \\;\\Rightarrow C-\\alpha^{\\lor}-\\mu^{\\lor} = 0\\\\\n",
    "\\frac{\\partial L}{\\partial \\xi_i^{\\land}} = 0 \\;\\Rightarrow C-\\alpha^{\\land}-\\mu^{\\land} = 0$$\n",
    "\n",
    "我们把上面4个式子带入$L(w,b,\\alpha^{\\lor}, \\alpha^{\\land}, \\xi_i^{\\lor}, \\xi_i^{\\land}, \\mu^{\\lor}, \\mu^{\\land})$消去$w,b,\\xi_i^{\\lor}, \\xi_i^{\\land}$\n",
    "\n",
    "整个推导过程同SVM线性分类一节类似，最终得到的对偶形式：\n",
    "$$\\underbrace{ min}_{\\alpha^{\\lor}, \\alpha^{\\land}}\\; \\frac{1}{2}\\sum\\limits_{i=1,j=1}^{m}(\\alpha_i^{\\land} - \\alpha_i^{\\lor})(\\alpha_j^{\\land} - \\alpha_j^{\\lor})K_{ij} + \\sum\\limits_{i=1}^{m}(\\epsilon-y_i)\\alpha_i^{\\land}+ (\\epsilon+y_i)\\alpha_i^{\\lor}\\\\\n",
    "s.t. \\; \\sum\\limits_{i=1}^{m}(\\alpha_i^{\\land} - \\alpha_i^{\\lor}) = 0\\\\\n",
    "0 < \\alpha_i^{\\lor} < C \\; (i =1,2,...m)\\\\\n",
    "0 < \\alpha_i^{\\land} < C \\; (i =1,2,...m)$$\n",
    "\n",
    "对于这个目标函数，我们可以用SMO算法来求出对应的$\\alpha^{\\lor}, \\alpha^{\\land}$，，进而求出我们的回归模型系数$w,b$。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SVM总结\n",
    "## 优点\n",
    "1. 解决高维特征的分类问题和回归问题很有效,在特征维度大于样本数时依然有很好的效果。\n",
    "2. 仅仅使用一部分支持向量来做超平面的决策，无需依赖全部数据。\n",
    "3. 有大量的核函数可以使用，从而可以很灵活的来解决各种非线性的分类回归问题。\n",
    "4. 样本量不是海量数据的时候，分类准确率高，泛化能力强。\n",
    "\n",
    "## 缺点\n",
    "1. 如果特征维度远远大于样本数，则SVM表现一般。\n",
    "2. SVM在样本量非常大，核函数映射维度非常高时，计算量过大，不太适合使用。\n",
    "3. 非线性问题的核函数的选择没有通用标准，难以选择一个合适的核函数。\n",
    "4. SVM对缺失数据敏感。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# scikit-learn SVM算法库使用概述\n",
    "## scikit-learn SVM算法库使用概述\n",
    "scikit-learn中SVM的算法库分为两类，一类是分类的算法库，包括SVC， NuSVC，和LinearSVC 3个类。另一类是回归算法库，包括SVR， NuSVR，和LinearSVR 3个类。相关的类都包裹在sklearn.svm模块之中。\n",
    "\n",
    "对于SVC， NuSVC，和LinearSVC 3个分类的类，SVC和 NuSVC差不多，区别仅仅在于对损失的度量方式不同，而LinearSVC从名字就可以看出，他是线性分类，也就是不支持各种低维到高维的核函数，仅仅支持线性核函数，对线性不可分的数据不能使用。\n",
    "\n",
    "对于SVR， NuSVR，和LinearSVR 3个回归的类，表述同上面的逻辑。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SVM分类算法库参数小结\n",
    "<table border=\"1\">\n",
    "<tbody>\n",
    "<tr>\n",
    "<td width=\"15%\"><strong>参数</strong></td>\n",
    "<td width=\"25%\"><strong>LinearSVC&nbsp;</strong></td>\n",
    "<td width=\"30%\"><strong>SVC</strong></td>\n",
    "<td width=\"30%\"><strong>NuSVC</strong></td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>惩罚系数C</td>\n",
    "<td colspan=\"2\">即为我们第二节中SVM分类模型原型形式和对偶形式中的惩罚系数C，默认为1，一般需要通过交叉验证来选择一个合适的C。一般来说，如果噪音点较多时，C需要小一些。</td>\n",
    "<td>NuSVC没有这个参数, 它通过另一个参数nu来控制训练集训练的错误率，等价于选择了一个C，让训练集训练后满足一个确定的错误率</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>nu</td>\n",
    "<td colspan=\"2\">LinearSVC 和SVC没有这个参数，LinearSVC 和SVC使用惩罚系数C来控制惩罚力度。&nbsp;</td>\n",
    "<td>nu代表训练集训练的错误率的上限，或者说支持向量的百分比下限，取值范围为(0,1],默认是0.5.它和惩罚系数C类似，都可以控制惩罚的力度。</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>核函数 kernel</td>\n",
    "<td>LinearSVC没有这个参数，LinearSVC限制了只能使用线性核函数</td>\n",
    "<td colspan=\"2\">\n",
    "<p>核函数有四种内置选择，第三节已经讲到：‘linear’即线性核函数, ‘poly’即多项式核函数, ‘rbf’即高斯核函数, ‘sigmoid’即sigmoid核函数。如果选择了这些核函数， 对应的核函数参数在后面有单独的参数需要调。默认是高斯核'rbf'。</p>\n",
    "<p>还有一种选择为\"precomputed\",即我们预先计算出所有的训练集和测试集的样本对应的Gram矩阵，这样<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-37-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>K</mi><mo stretchy=&quot;false&quot;>(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=&quot;false&quot;>)</mo></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-836\" style=\"width: 3.781em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 3.129em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.257em 1003.03em 2.611em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-837\"><span class=\"mi\" id=\"MathJax-Span-838\" style=\"font-family: MathJax_Math-italic;\">K<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.04em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-839\" style=\"font-family: MathJax_Main;\">(</span><span class=\"mi\" id=\"MathJax-Span-840\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mo\" id=\"MathJax-Span-841\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-842\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">z<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-843\" style=\"font-family: MathJax_Main;\">)</span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.374em; border-left: 0px solid; width: 0px; height: 1.353em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>K</mi><mo stretchy=\"false\">(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=\"false\">)</mo></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-37\">K(x,z)</script>直接在对应的Gram矩阵中找对应的位置的值。</p>\n",
    "<p>当然我们也可以自定义核函数， 由于我没有用过自定义核函数，这里就不多讲了。</p>\n",
    "</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>正则化参数penalty&nbsp;</td>\n",
    "<td>仅仅对线性拟合有意义，可以选择‘l1’即L1正则化 或者 ‘l2’即L2正则化。默认是L2正则化，如果我们需要产生稀疏话的系数的时候，可以选L1正则化,这和线性回归里面的Lasso回归类似。</td>\n",
    "<td colspan=\"2\">SVC和NuSVC没有这个参数</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>是否用对偶形式优化dual&nbsp;</td>\n",
    "<td>这是一个布尔变量，控制是否使用对偶形式来优化算法，默认是True,即采用上面第二节的分类算法对偶形式来优化算法。如果我们的样本量比特征数多，此时采用对偶形式计算量较大，推荐dual设置为False，即采用原始形式优化</td>\n",
    "<td colspan=\"2\">SVC和NuSVC没有这个参数&nbsp;&nbsp;</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>核函数参数degree</td>\n",
    "<td>&nbsp;LinearSVC没有这个参数，LinearSVC限制了只能使用线性核函数</td>\n",
    "<td colspan=\"2\">如果我们在kernel参数使用了多项式核函数 'poly'，那么我们就需要对这个参数进行调参。这个参数对应<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-38-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>K</mi><mo stretchy=&quot;false&quot;>(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=&quot;false&quot;>)</mo><mo>=</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#xFF08;</mo></mrow><mi>&amp;#x03B3;</mi><mi>x</mi><mo>&amp;#x2219;</mo><mi>z</mi><mo>+</mo><mi>r</mi><msup><mo stretchy=&quot;false&quot;>)</mo><mi>d</mi></msup></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-844\" style=\"width: 12.4em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 10.213em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.153em 1010.21em 2.611em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-845\"><span class=\"mi\" id=\"MathJax-Span-846\" style=\"font-family: MathJax_Math-italic;\">K<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.04em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-847\" style=\"font-family: MathJax_Main;\">(</span><span class=\"mi\" id=\"MathJax-Span-848\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mo\" id=\"MathJax-Span-849\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-850\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">z<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-851\" style=\"font-family: MathJax_Main;\">)</span><span class=\"mo\" id=\"MathJax-Span-852\" style=\"font-family: MathJax_Main; padding-left: 0.278em;\">=</span><span class=\"texatom\" id=\"MathJax-Span-853\" style=\"padding-left: 0.278em;\"><span class=\"mrow\" id=\"MathJax-Span-854\"><span class=\"mo\" id=\"MathJax-Span-855\"><span style=\"font-family: STIXGeneral, &quot;Arial Unicode MS&quot;, serif; font-size: 83%; font-style: normal; font-weight: normal;\">（</span></span></span></span><span class=\"mi\" id=\"MathJax-Span-856\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span><span class=\"mi\" id=\"MathJax-Span-857\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mo\" id=\"MathJax-Span-858\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">∙</span><span class=\"mi\" id=\"MathJax-Span-859\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">z<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-860\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">+</span><span class=\"mi\" id=\"MathJax-Span-861\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">r</span><span class=\"msubsup\" id=\"MathJax-Span-862\"><span style=\"display: inline-block; position: relative; width: 0.834em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.087em 1000.29em 4.441em -1000em); top: -4.014em; left: 0em;\"><span class=\"mo\" id=\"MathJax-Span-863\" style=\"font-family: MathJax_Main;\">)</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; top: -4.377em; left: 0.389em;\"><span class=\"mi\" id=\"MathJax-Span-864\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">d<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.002em;\"></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.374em; border-left: 0px solid; width: 0px; height: 1.478em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>K</mi><mo stretchy=\"false\">(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=\"false\">)</mo><mo>=</mo><mrow class=\"MJX-TeXAtom-ORD\"><mo>（</mo></mrow><mi>γ</mi><mi>x</mi><mo>∙</mo><mi>z</mi><mo>+</mo><mi>r</mi><msup><mo stretchy=\"false\">)</mo><mi>d</mi></msup></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-38\">K(x, z) = （\\gamma x \\bullet z  + r)^d</script>中的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-39-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>d</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-865\" style=\"width: 0.652em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.531em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.254em 1000.53em 2.312em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-866\"><span class=\"mi\" id=\"MathJax-Span-867\" style=\"font-family: MathJax_Math-italic;\">d<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.084em; border-left: 0px solid; width: 0px; height: 0.995em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>d</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-39\">d</script>。默认是3。一般需要通过交叉验证选择一组合适的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-40-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03B3;</mi><mo>,</mo><mi>r</mi><mo>,</mo><mi>d</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-868\" style=\"width: 2.955em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 2.42em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.313em 1002.42em 2.577em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-869\"><span class=\"mi\" id=\"MathJax-Span-870\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-871\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-872\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">r</span><span class=\"mo\" id=\"MathJax-Span-873\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-874\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">d<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.333em; border-left: 0px solid; width: 0px; height: 1.244em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>γ</mi><mo>,</mo><mi>r</mi><mo>,</mo><mi>d</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-40\">\\gamma, r, d</script></td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>核函数参数gamma&nbsp;</td>\n",
    "<td>&nbsp;&nbsp;LinearSVC没有这个参数，LinearSVC限制了只能使用线性核函数</td>\n",
    "<td colspan=\"2\">\n",
    "<p>如果我们在kernel参数使用了多项式核函数 'poly'，高斯核函数‘rbf’, 或者sigmoid核函数，那么我们就需要对这个参数进行调参。</p>\n",
    "<p>多项式核函数中这个参数对应<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-41-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>K</mi><mo stretchy=&quot;false&quot;>(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=&quot;false&quot;>)</mo><mo>=</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#xFF08;</mo></mrow><mi>&amp;#x03B3;</mi><mi>x</mi><mo>&amp;#x2219;</mo><mi>z</mi><mo>+</mo><mi>r</mi><msup><mo stretchy=&quot;false&quot;>)</mo><mi>d</mi></msup></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-875\" style=\"width: 12.4em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 10.213em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.153em 1010.21em 2.611em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-876\"><span class=\"mi\" id=\"MathJax-Span-877\" style=\"font-family: MathJax_Math-italic;\">K<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.04em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-878\" style=\"font-family: MathJax_Main;\">(</span><span class=\"mi\" id=\"MathJax-Span-879\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mo\" id=\"MathJax-Span-880\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-881\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">z<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-882\" style=\"font-family: MathJax_Main;\">)</span><span class=\"mo\" id=\"MathJax-Span-883\" style=\"font-family: MathJax_Main; padding-left: 0.278em;\">=</span><span class=\"texatom\" id=\"MathJax-Span-884\" style=\"padding-left: 0.278em;\"><span class=\"mrow\" id=\"MathJax-Span-885\"><span class=\"mo\" id=\"MathJax-Span-886\"><span style=\"font-family: STIXGeneral, &quot;Arial Unicode MS&quot;, serif; font-size: 83%; font-style: normal; font-weight: normal;\">（</span></span></span></span><span class=\"mi\" id=\"MathJax-Span-887\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span><span class=\"mi\" id=\"MathJax-Span-888\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mo\" id=\"MathJax-Span-889\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">∙</span><span class=\"mi\" id=\"MathJax-Span-890\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">z<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-891\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">+</span><span class=\"mi\" id=\"MathJax-Span-892\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">r</span><span class=\"msubsup\" id=\"MathJax-Span-893\"><span style=\"display: inline-block; position: relative; width: 0.834em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.087em 1000.29em 4.441em -1000em); top: -4.014em; left: 0em;\"><span class=\"mo\" id=\"MathJax-Span-894\" style=\"font-family: MathJax_Main;\">)</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; top: -4.377em; left: 0.389em;\"><span class=\"mi\" id=\"MathJax-Span-895\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">d<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.002em;\"></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.374em; border-left: 0px solid; width: 0px; height: 1.478em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>K</mi><mo stretchy=\"false\">(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=\"false\">)</mo><mo>=</mo><mrow class=\"MJX-TeXAtom-ORD\"><mo>（</mo></mrow><mi>γ</mi><mi>x</mi><mo>∙</mo><mi>z</mi><mo>+</mo><mi>r</mi><msup><mo stretchy=\"false\">)</mo><mi>d</mi></msup></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-41\">K(x, z) = （\\gamma x \\bullet z  + r)^d</script>中的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-42-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03B3;</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-896\" style=\"width: 0.652em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.531em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.507em 1000.53em 2.518em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-897\"><span class=\"mi\" id=\"MathJax-Span-898\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.333em; border-left: 0px solid; width: 0px; height: 0.938em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>γ</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-42\">\\gamma</script>。一般需要通过交叉验证选择一组合适的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-43-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03B3;</mi><mo>,</mo><mi>r</mi><mo>,</mo><mi>d</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-899\" style=\"width: 2.955em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 2.42em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.313em 1002.42em 2.577em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-900\"><span class=\"mi\" id=\"MathJax-Span-901\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-902\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-903\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">r</span><span class=\"mo\" id=\"MathJax-Span-904\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-905\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">d<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.333em; border-left: 0px solid; width: 0px; height: 1.244em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>γ</mi><mo>,</mo><mi>r</mi><mo>,</mo><mi>d</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-43\">\\gamma, r, d</script>&nbsp;</p>\n",
    "<p>高斯核函数中这个参数对应<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-44-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>K</mi><mo stretchy=&quot;false&quot;>(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=&quot;false&quot;>)</mo><mo>=</mo><mi>e</mi><mi>x</mi><mi>p</mi><mo stretchy=&quot;false&quot;>(</mo><mo>&amp;#x2212;</mo><mi>&amp;#x03B3;</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo stretchy=&quot;false&quot;>|</mo></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo stretchy=&quot;false&quot;>|</mo></mrow><mi>x</mi><mo>&amp;#x2212;</mo><mi>z</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo stretchy=&quot;false&quot;>|</mo></mrow><msup><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo stretchy=&quot;false&quot;>|</mo></mrow><mn>2</mn></msup><mo stretchy=&quot;false&quot;>)</mo></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-906\" style=\"width: 14.407em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 11.865em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.059em 1011.77em 2.611em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-907\"><span class=\"mi\" id=\"MathJax-Span-908\" style=\"font-family: MathJax_Math-italic;\">K<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.04em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-909\" style=\"font-family: MathJax_Main;\">(</span><span class=\"mi\" id=\"MathJax-Span-910\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mo\" id=\"MathJax-Span-911\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-912\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">z<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-913\" style=\"font-family: MathJax_Main;\">)</span><span class=\"mo\" id=\"MathJax-Span-914\" style=\"font-family: MathJax_Main; padding-left: 0.278em;\">=</span><span class=\"mi\" id=\"MathJax-Span-915\" style=\"font-family: MathJax_Math-italic; padding-left: 0.278em;\">e</span><span class=\"mi\" id=\"MathJax-Span-916\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mi\" id=\"MathJax-Span-917\" style=\"font-family: MathJax_Math-italic;\">p</span><span class=\"mo\" id=\"MathJax-Span-918\" style=\"font-family: MathJax_Main;\">(</span><span class=\"mo\" id=\"MathJax-Span-919\" style=\"font-family: MathJax_Main;\">−</span><span class=\"mi\" id=\"MathJax-Span-920\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span><span class=\"texatom\" id=\"MathJax-Span-921\"><span class=\"mrow\" id=\"MathJax-Span-922\"><span class=\"mo\" id=\"MathJax-Span-923\" style=\"font-family: MathJax_Main;\">|</span></span></span><span class=\"texatom\" id=\"MathJax-Span-924\"><span class=\"mrow\" id=\"MathJax-Span-925\"><span class=\"mo\" id=\"MathJax-Span-926\" style=\"font-family: MathJax_Main;\">|</span></span></span><span class=\"mi\" id=\"MathJax-Span-927\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mo\" id=\"MathJax-Span-928\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">−</span><span class=\"mi\" id=\"MathJax-Span-929\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">z<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span><span class=\"texatom\" id=\"MathJax-Span-930\"><span class=\"mrow\" id=\"MathJax-Span-931\"><span class=\"mo\" id=\"MathJax-Span-932\" style=\"font-family: MathJax_Main;\">|</span></span></span><span class=\"msubsup\" id=\"MathJax-Span-933\"><span style=\"display: inline-block; position: relative; width: 0.707em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.087em 1000.16em 4.44em -1000em); top: -4.014em; left: 0em;\"><span class=\"texatom\" id=\"MathJax-Span-934\"><span class=\"mrow\" id=\"MathJax-Span-935\"><span class=\"mo\" id=\"MathJax-Span-936\" style=\"font-family: MathJax_Main;\">|</span></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; top: -4.491em; left: 0.278em;\"><span class=\"mn\" id=\"MathJax-Span-937\" style=\"font-size: 70.7%; font-family: MathJax_Main;\">2</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span><span class=\"mo\" id=\"MathJax-Span-938\" style=\"font-family: MathJax_Main;\">)</span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.374em; border-left: 0px solid; width: 0px; height: 1.592em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>K</mi><mo stretchy=\"false\">(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=\"false\">)</mo><mo>=</mo><mi>e</mi><mi>x</mi><mi>p</mi><mo stretchy=\"false\">(</mo><mo>−</mo><mi>γ</mi><mrow class=\"MJX-TeXAtom-ORD\"><mo stretchy=\"false\">|</mo></mrow><mrow class=\"MJX-TeXAtom-ORD\"><mo stretchy=\"false\">|</mo></mrow><mi>x</mi><mo>−</mo><mi>z</mi><mrow class=\"MJX-TeXAtom-ORD\"><mo stretchy=\"false\">|</mo></mrow><msup><mrow class=\"MJX-TeXAtom-ORD\"><mo stretchy=\"false\">|</mo></mrow><mn>2</mn></msup><mo stretchy=\"false\">)</mo></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-44\">K(x, z) = exp(-\\gamma||x-z||^2)</script>中的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-45-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03B3;</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-939\" style=\"width: 0.652em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.531em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.507em 1000.53em 2.518em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-940\"><span class=\"mi\" id=\"MathJax-Span-941\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.333em; border-left: 0px solid; width: 0px; height: 0.938em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>γ</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-45\">\\gamma</script>。一般需要通过交叉验证选择合适的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-46-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03B3;</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-942\" style=\"width: 0.652em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.531em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.507em 1000.53em 2.518em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-943\"><span class=\"mi\" id=\"MathJax-Span-944\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.333em; border-left: 0px solid; width: 0px; height: 0.938em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>γ</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-46\">\\gamma</script></p>\n",
    "<p>sigmoid核函数中这个参数对应<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-47-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>K</mi><mo stretchy=&quot;false&quot;>(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=&quot;false&quot;>)</mo><mo>=</mo><mi>t</mi><mi>a</mi><mi>n</mi><mi>h</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#xFF08;</mo></mrow><mi>&amp;#x03B3;</mi><mi>x</mi><mo>&amp;#x2219;</mo><mi>z</mi><mo>+</mo><mi>r</mi><mo stretchy=&quot;false&quot;>)</mo></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-945\" style=\"width: 14.407em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 11.865em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.207em 1011.77em 2.611em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-946\"><span class=\"mi\" id=\"MathJax-Span-947\" style=\"font-family: MathJax_Math-italic;\">K<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.04em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-948\" style=\"font-family: MathJax_Main;\">(</span><span class=\"mi\" id=\"MathJax-Span-949\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mo\" id=\"MathJax-Span-950\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-951\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">z<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-952\" style=\"font-family: MathJax_Main;\">)</span><span class=\"mo\" id=\"MathJax-Span-953\" style=\"font-family: MathJax_Main; padding-left: 0.278em;\">=</span><span class=\"mi\" id=\"MathJax-Span-954\" style=\"font-family: MathJax_Math-italic; padding-left: 0.278em;\">t</span><span class=\"mi\" id=\"MathJax-Span-955\" style=\"font-family: MathJax_Math-italic;\">a</span><span class=\"mi\" id=\"MathJax-Span-956\" style=\"font-family: MathJax_Math-italic;\">n</span><span class=\"mi\" id=\"MathJax-Span-957\" style=\"font-family: MathJax_Math-italic;\">h</span><span class=\"texatom\" id=\"MathJax-Span-958\"><span class=\"mrow\" id=\"MathJax-Span-959\"><span class=\"mo\" id=\"MathJax-Span-960\"><span style=\"font-family: STIXGeneral, &quot;Arial Unicode MS&quot;, serif; font-size: 83%; font-style: normal; font-weight: normal;\">（</span></span></span></span><span class=\"mi\" id=\"MathJax-Span-961\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span><span class=\"mi\" id=\"MathJax-Span-962\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mo\" id=\"MathJax-Span-963\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">∙</span><span class=\"mi\" id=\"MathJax-Span-964\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">z<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-965\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">+</span><span class=\"mi\" id=\"MathJax-Span-966\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">r</span><span class=\"mo\" id=\"MathJax-Span-967\" style=\"font-family: MathJax_Main;\">)</span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.374em; border-left: 0px solid; width: 0px; height: 1.413em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>K</mi><mo stretchy=\"false\">(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=\"false\">)</mo><mo>=</mo><mi>t</mi><mi>a</mi><mi>n</mi><mi>h</mi><mrow class=\"MJX-TeXAtom-ORD\"><mo>（</mo></mrow><mi>γ</mi><mi>x</mi><mo>∙</mo><mi>z</mi><mo>+</mo><mi>r</mi><mo stretchy=\"false\">)</mo></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-47\">K(x, z) = tanh（\\gamma x \\bullet z  + r)</script>中的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-48-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03B3;</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-968\" style=\"width: 0.652em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.531em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.507em 1000.53em 2.518em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-969\"><span class=\"mi\" id=\"MathJax-Span-970\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.333em; border-left: 0px solid; width: 0px; height: 0.938em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>γ</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-48\">\\gamma</script>。一般需要通过交叉验证选择一组合适的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-49-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03B3;</mi><mo>,</mo><mi>r</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-971\" style=\"width: 1.715em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 1.417em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.565em 1001.4em 2.577em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-972\"><span class=\"mi\" id=\"MathJax-Span-973\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-974\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-975\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">r</span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.333em; border-left: 0px solid; width: 0px; height: 0.939em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>γ</mi><mo>,</mo><mi>r</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-49\">\\gamma, r</script>&nbsp;</p>\n",
    "<p><span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-50-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03B3;</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-976\" style=\"width: 0.652em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.531em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.507em 1000.53em 2.518em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-977\"><span class=\"mi\" id=\"MathJax-Span-978\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.333em; border-left: 0px solid; width: 0px; height: 0.938em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>γ</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-50\">\\gamma</script>默认为'auto',即<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-51-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mfrac><mn>1</mn><mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#x7279;</mo></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#x5F81;</mo></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#x7EF4;</mo></mrow><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#x5EA6;</mo></mrow></mrow></mfrac></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-979\" style=\"width: 3.309em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 2.715em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.056em 1002.72em 2.93em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-980\"><span class=\"mfrac\" id=\"MathJax-Span-981\"><span style=\"display: inline-block; position: relative; width: 2.481em; height: 0px; margin-right: 0.12em; margin-left: 0.12em;\"><span style=\"position: absolute; clip: rect(3.366em 1000.3em 4.191em -1000em); top: -4.435em; left: 50%; margin-left: -0.177em;\"><span class=\"mn\" id=\"MathJax-Span-982\" style=\"font-size: 70.7%; font-family: MathJax_Main;\">1</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; clip: rect(1.382em 1002.36em 2.444em -1000em); top: -1.639em; left: 50%; margin-left: -1.181em;\"><span class=\"mrow\" id=\"MathJax-Span-983\"><span class=\"texatom\" id=\"MathJax-Span-984\"><span class=\"mrow\" id=\"MathJax-Span-985\"><span class=\"mo\" id=\"MathJax-Span-986\" style=\"font-size: 70.7%;\"><span style=\"font-family: STIXGeneral, &quot;Arial Unicode MS&quot;, serif; font-size: 83%; font-style: normal; font-weight: normal;\">特</span></span></span></span><span class=\"texatom\" id=\"MathJax-Span-987\"><span class=\"mrow\" id=\"MathJax-Span-988\"><span class=\"mo\" id=\"MathJax-Span-989\" style=\"font-size: 70.7%;\"><span style=\"font-family: STIXGeneral, &quot;Arial Unicode MS&quot;, serif; font-size: 83%; font-style: normal; font-weight: normal;\">征</span></span></span></span><span class=\"texatom\" id=\"MathJax-Span-990\"><span class=\"mrow\" id=\"MathJax-Span-991\"><span class=\"mo\" id=\"MathJax-Span-992\" style=\"font-size: 70.7%;\"><span style=\"font-family: STIXGeneral, &quot;Arial Unicode MS&quot;, serif; font-size: 83%; font-style: normal; font-weight: normal;\">维</span></span></span></span><span class=\"texatom\" id=\"MathJax-Span-993\"><span class=\"mrow\" id=\"MathJax-Span-994\"><span class=\"mo\" id=\"MathJax-Span-995\" style=\"font-size: 70.7%;\"><span style=\"font-family: STIXGeneral, &quot;Arial Unicode MS&quot;, serif; font-size: 83%; font-style: normal; font-weight: normal;\">度</span></span></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span><span style=\"position: absolute; clip: rect(0.812em 1002.48em 1.24em -1000em); top: -1.283em; left: 0em;\"><span style=\"display: inline-block; overflow: hidden; vertical-align: 0em; border-top: 1.3px solid; width: 2.481em; height: 0px;\"></span><span style=\"display: inline-block; width: 0px; height: 1.063em;\"></span></span></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.831em; border-left: 0px solid; width: 0px; height: 1.981em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mfrac><mn>1</mn><mrow><mrow class=\"MJX-TeXAtom-ORD\"><mo>特</mo></mrow><mrow class=\"MJX-TeXAtom-ORD\"><mo>征</mo></mrow><mrow class=\"MJX-TeXAtom-ORD\"><mo>维</mo></mrow><mrow class=\"MJX-TeXAtom-ORD\"><mo>度</mo></mrow></mrow></mfrac></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-51\">\\frac{1}{特征维度}</script></p>\n",
    "</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>&nbsp;核函数参数coef0</td>\n",
    "<td>&nbsp;LinearSVC没有这个参数，LinearSVC限制了只能使用线性核函数</td>\n",
    "<td colspan=\"2\">&nbsp;\n",
    "<p>如果我们在kernel参数使用了多项式核函数 'poly'，或者sigmoid核函数，那么我们就需要对这个参数进行调参。</p>\n",
    "<p>多项式核函数中这个参数对应<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-52-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>K</mi><mo stretchy=&quot;false&quot;>(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=&quot;false&quot;>)</mo><mo>=</mo><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#xFF08;</mo></mrow><mi>&amp;#x03B3;</mi><mi>x</mi><mo>&amp;#x2219;</mo><mi>z</mi><mo>+</mo><mi>r</mi><msup><mo stretchy=&quot;false&quot;>)</mo><mi>d</mi></msup></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-996\" style=\"width: 12.4em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 10.213em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.153em 1010.21em 2.611em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-997\"><span class=\"mi\" id=\"MathJax-Span-998\" style=\"font-family: MathJax_Math-italic;\">K<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.04em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-999\" style=\"font-family: MathJax_Main;\">(</span><span class=\"mi\" id=\"MathJax-Span-1000\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mo\" id=\"MathJax-Span-1001\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-1002\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">z<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-1003\" style=\"font-family: MathJax_Main;\">)</span><span class=\"mo\" id=\"MathJax-Span-1004\" style=\"font-family: MathJax_Main; padding-left: 0.278em;\">=</span><span class=\"texatom\" id=\"MathJax-Span-1005\" style=\"padding-left: 0.278em;\"><span class=\"mrow\" id=\"MathJax-Span-1006\"><span class=\"mo\" id=\"MathJax-Span-1007\"><span style=\"font-family: STIXGeneral, &quot;Arial Unicode MS&quot;, serif; font-size: 83%; font-style: normal; font-weight: normal;\">（</span></span></span></span><span class=\"mi\" id=\"MathJax-Span-1008\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span><span class=\"mi\" id=\"MathJax-Span-1009\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mo\" id=\"MathJax-Span-1010\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">∙</span><span class=\"mi\" id=\"MathJax-Span-1011\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">z<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-1012\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">+</span><span class=\"mi\" id=\"MathJax-Span-1013\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">r</span><span class=\"msubsup\" id=\"MathJax-Span-1014\"><span style=\"display: inline-block; position: relative; width: 0.834em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.087em 1000.29em 4.441em -1000em); top: -4.014em; left: 0em;\"><span class=\"mo\" id=\"MathJax-Span-1015\" style=\"font-family: MathJax_Main;\">)</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; top: -4.377em; left: 0.389em;\"><span class=\"mi\" id=\"MathJax-Span-1016\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">d<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.002em;\"></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.374em; border-left: 0px solid; width: 0px; height: 1.478em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>K</mi><mo stretchy=\"false\">(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=\"false\">)</mo><mo>=</mo><mrow class=\"MJX-TeXAtom-ORD\"><mo>（</mo></mrow><mi>γ</mi><mi>x</mi><mo>∙</mo><mi>z</mi><mo>+</mo><mi>r</mi><msup><mo stretchy=\"false\">)</mo><mi>d</mi></msup></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-52\">K(x, z) = （\\gamma x \\bullet z  + r)^d</script>中的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-53-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>r</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1017\" style=\"width: 0.534em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.413em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.506em 1000.39em 2.313em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1018\"><span class=\"mi\" id=\"MathJax-Span-1019\" style=\"font-family: MathJax_Math-italic;\">r</span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.085em; border-left: 0px solid; width: 0px; height: 0.691em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>r</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-53\">r</script>。一般需要通过交叉验证选择一组合适的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-54-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03B3;</mi><mo>,</mo><mi>r</mi><mo>,</mo><mi>d</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1020\" style=\"width: 2.955em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 2.42em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.313em 1002.42em 2.577em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1021\"><span class=\"mi\" id=\"MathJax-Span-1022\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-1023\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-1024\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">r</span><span class=\"mo\" id=\"MathJax-Span-1025\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-1026\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">d<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.333em; border-left: 0px solid; width: 0px; height: 1.244em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>γ</mi><mo>,</mo><mi>r</mi><mo>,</mo><mi>d</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-54\">\\gamma, r, d</script>&nbsp;</p>\n",
    "<p>sigmoid核函数中这个参数对应<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-55-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>K</mi><mo stretchy=&quot;false&quot;>(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=&quot;false&quot;>)</mo><mo>=</mo><mi>t</mi><mi>a</mi><mi>n</mi><mi>h</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#xFF08;</mo></mrow><mi>&amp;#x03B3;</mi><mi>x</mi><mo>&amp;#x2219;</mo><mi>z</mi><mo>+</mo><mi>r</mi><mo stretchy=&quot;false&quot;>)</mo></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1027\" style=\"width: 14.407em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 11.865em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.207em 1011.77em 2.611em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1028\"><span class=\"mi\" id=\"MathJax-Span-1029\" style=\"font-family: MathJax_Math-italic;\">K<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.04em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-1030\" style=\"font-family: MathJax_Main;\">(</span><span class=\"mi\" id=\"MathJax-Span-1031\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mo\" id=\"MathJax-Span-1032\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-1033\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">z<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-1034\" style=\"font-family: MathJax_Main;\">)</span><span class=\"mo\" id=\"MathJax-Span-1035\" style=\"font-family: MathJax_Main; padding-left: 0.278em;\">=</span><span class=\"mi\" id=\"MathJax-Span-1036\" style=\"font-family: MathJax_Math-italic; padding-left: 0.278em;\">t</span><span class=\"mi\" id=\"MathJax-Span-1037\" style=\"font-family: MathJax_Math-italic;\">a</span><span class=\"mi\" id=\"MathJax-Span-1038\" style=\"font-family: MathJax_Math-italic;\">n</span><span class=\"mi\" id=\"MathJax-Span-1039\" style=\"font-family: MathJax_Math-italic;\">h</span><span class=\"texatom\" id=\"MathJax-Span-1040\"><span class=\"mrow\" id=\"MathJax-Span-1041\"><span class=\"mo\" id=\"MathJax-Span-1042\"><span style=\"font-family: STIXGeneral, &quot;Arial Unicode MS&quot;, serif; font-size: 83%; font-style: normal; font-weight: normal;\">（</span></span></span></span><span class=\"mi\" id=\"MathJax-Span-1043\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span><span class=\"mi\" id=\"MathJax-Span-1044\" style=\"font-family: MathJax_Math-italic;\">x</span><span class=\"mo\" id=\"MathJax-Span-1045\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">∙</span><span class=\"mi\" id=\"MathJax-Span-1046\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">z<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.003em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-1047\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">+</span><span class=\"mi\" id=\"MathJax-Span-1048\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">r</span><span class=\"mo\" id=\"MathJax-Span-1049\" style=\"font-family: MathJax_Main;\">)</span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.374em; border-left: 0px solid; width: 0px; height: 1.413em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>K</mi><mo stretchy=\"false\">(</mo><mi>x</mi><mo>,</mo><mi>z</mi><mo stretchy=\"false\">)</mo><mo>=</mo><mi>t</mi><mi>a</mi><mi>n</mi><mi>h</mi><mrow class=\"MJX-TeXAtom-ORD\"><mo>（</mo></mrow><mi>γ</mi><mi>x</mi><mo>∙</mo><mi>z</mi><mo>+</mo><mi>r</mi><mo stretchy=\"false\">)</mo></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-55\">K(x, z) = tanh（\\gamma x \\bullet z  + r)</script>中的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-56-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>r</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1050\" style=\"width: 0.534em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.413em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.506em 1000.39em 2.313em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1051\"><span class=\"mi\" id=\"MathJax-Span-1052\" style=\"font-family: MathJax_Math-italic;\">r</span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.085em; border-left: 0px solid; width: 0px; height: 0.691em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>r</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-56\">r</script>。一般需要通过交叉验证选择一组合适的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-57-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03B3;</mi><mo>,</mo><mi>r</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1053\" style=\"width: 1.715em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 1.417em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.565em 1001.4em 2.577em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1054\"><span class=\"mi\" id=\"MathJax-Span-1055\" style=\"font-family: MathJax_Math-italic;\">γ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.025em;\"></span></span><span class=\"mo\" id=\"MathJax-Span-1056\" style=\"font-family: MathJax_Main;\">,</span><span class=\"mi\" id=\"MathJax-Span-1057\" style=\"font-family: MathJax_Math-italic; padding-left: 0.167em;\">r</span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.333em; border-left: 0px solid; width: 0px; height: 0.939em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>γ</mi><mo>,</mo><mi>r</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-57\">\\gamma, r</script>&nbsp;</p>\n",
    "<p>coef0默认为0</p>\n",
    "&nbsp;</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>&nbsp;样本权重class_weight</td>\n",
    "<td colspan=\"3\">&nbsp;指定样本各类别的的权重，主要是为了防止训练集某些类别的样本过多，导致训练的决策过于偏向这些类别。这里可以自己指定各个样本的权重，或者用“balanced”，如果使用“balanced”，则算法会自己计算权重，样本量少的类别所对应的样本权重会高。当然，如果你的样本类别分布没有明显的偏倚，则可以不管这个参数，选择默认的\"None\"</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>分类决策decision_function_shape</td>\n",
    "<td>&nbsp;LinearSVC没有这个参数，使用multi_class参数替代。</td>\n",
    "<td colspan=\"2\">可以选择'ovo'或者‘ovr’.目前0.18版本默认是'ovo'.0.19版本将是'ovr'\n",
    "<p>OvR(one ve rest)的思想很简单，无论你是多少元分类，我们都可以看做二元分类。具体做法是，对于第K类的分类决策，我们把所有第K类的样本作为正例，除了第K类样本以外的所有样本都作为负例，然后在上面做二元分类，得到第K类的分类模型。其他类的分类模型获得以此类推。</p>\n",
    "<p>OvO(one-vs-one)则是每次每次在所有的T类样本里面选择两类样本出来，不妨记为T1类和T2类，把所有的输出为T1和T2的样本放在一起，把T1作为正例，T2作为负例，进行二元分类，得到模型参数。我们一共需要T(T-1)/2次分类。</p>\n",
    "<p>从上面的描述可以看出OvR相对简单，但分类效果相对略差（这里指大多数样本分布情况，某些样本分布下OvR可能更好）。而OvO分类相对精确，但是分类速度没有OvR快。一般建议使用OvO以达到较好的分类效果。</p>\n",
    "</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>分类决策multi_class</td>\n",
    "<td>\n",
    "<p>可以选择 ‘ovr’ 或者 ‘crammer_singer’&nbsp;</p>\n",
    "<p>‘ovr’和SVC和nuSVC中的decision_function_shape对应的‘ovr’类似。</p>\n",
    "<p>'crammer_singer'是一种改良版的'ovr'，说是改良，但是没有比’ovr‘好，一般在应用中都不建议使用。</p>\n",
    "</td>\n",
    "<td colspan=\"2\">SVC和nuSVC没有这个参数，使用decision_function_shape参数替代。&nbsp;</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>缓存大小cache_size</td>\n",
    "<td>\n",
    "<p>LinearSVC计算量不大，因此不需要这个参数</p>\n",
    "</td>\n",
    "<td colspan=\"2\">在大样本的时候，缓存大小会影响训练速度，因此如果机器内存大，推荐用500MB甚至1000MB。默认是200，即200MB.</td>\n",
    "</tr>\n",
    "</tbody>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SVM回归算法库参数小结\n",
    "<table border=\"0\">\n",
    "<tbody>\n",
    "<tr>\n",
    "<td width=\"15%\"><strong>参数</strong></td>\n",
    "<td width=\"25%\"><strong>LinearSVR</strong></td>\n",
    "<td width=\"30%\"><strong>SVR</strong></td>\n",
    "<td width=\"30%\"><strong>nuSVR</strong></td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>惩罚系数C</td>\n",
    "<td colspan=\"3\">即为我们第二节中SVM分类模型原型形式和对偶形式中的惩罚系数C，默认为1，一般需要通过交叉验证来选择一个合适的C。一般来说，如果噪音点较多时，C需要小一些。大家可能注意到在分类模型里面，nuSVC使用了nu这个等价的参数控制错误率，就没有使用C，为什么我们nuSVR仍然有这个参数呢，不是重复了吗？这里的原因在回归模型里面，我们除了惩罚系数C还有还有一个距离误差<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-58-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03F5;</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1058\" style=\"width: 0.534em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.413em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.517em 1000.39em 2.313em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1059\"><span class=\"mi\" id=\"MathJax-Span-1060\" style=\"font-family: MathJax_Math-italic;\">ϵ</span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.085em; border-left: 0px solid; width: 0px; height: 0.678em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>ϵ</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-58\">\\epsilon</script>来控制损失度量，因此仅仅一个nu不能等同于C.也就是说回归错误率是惩罚系数C和距离误差<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-59-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03F5;</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1061\" style=\"width: 0.534em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.413em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.517em 1000.39em 2.313em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1062\"><span class=\"mi\" id=\"MathJax-Span-1063\" style=\"font-family: MathJax_Math-italic;\">ϵ</span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.085em; border-left: 0px solid; width: 0px; height: 0.678em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>ϵ</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-59\">\\epsilon</script>共同作用的结果。后面我们可以看到nuSVR中nu的作用。</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>nu</td>\n",
    "<td colspan=\"2\">LinearSVR 和SVR没有这个参数，用<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-60-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03F5;</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1064\" style=\"width: 0.534em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.413em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.517em 1000.39em 2.313em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1065\"><span class=\"mi\" id=\"MathJax-Span-1066\" style=\"font-family: MathJax_Math-italic;\">ϵ</span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.085em; border-left: 0px solid; width: 0px; height: 0.678em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>ϵ</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-60\">\\epsilon</script>控制错误率</td>\n",
    "<td>nu代表训练集训练的错误率的上限，或者说支持向量的百分比下限，取值范围为(0,1],默认是0.5.通过选择不同的错误率可以得到不同的距离误差<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-61-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03F5;</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1067\" style=\"width: 0.534em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.413em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.517em 1000.39em 2.313em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1068\"><span class=\"mi\" id=\"MathJax-Span-1069\" style=\"font-family: MathJax_Math-italic;\">ϵ</span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.085em; border-left: 0px solid; width: 0px; height: 0.678em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>ϵ</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-61\">\\epsilon</script>。也就是说这里的nu的使用和LinearSVR 和SVR的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-62-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03F5;</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1070\" style=\"width: 0.534em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.413em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.517em 1000.39em 2.313em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1071\"><span class=\"mi\" id=\"MathJax-Span-1072\" style=\"font-family: MathJax_Math-italic;\">ϵ</span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.085em; border-left: 0px solid; width: 0px; height: 0.678em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>ϵ</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-62\">\\epsilon</script>参数等价。</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>距离误差epsilon</td>\n",
    "<td colspan=\"2\">即我们第二节回归模型中的<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-63-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mi>&amp;#x03F5;</mi></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1073\" style=\"width: 0.534em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 0.413em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.517em 1000.39em 2.313em -1000em); top: -2.125em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1074\"><span class=\"mi\" id=\"MathJax-Span-1075\" style=\"font-family: MathJax_Math-italic;\">ϵ</span></span><span style=\"display: inline-block; width: 0px; height: 2.125em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.085em; border-left: 0px solid; width: 0px; height: 0.678em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mi>ϵ</mi></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-63\">\\epsilon</script>，训练集中的样本需满足<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-64-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mo>&amp;#x2212;</mo><mi>&amp;#x03F5;</mi><mo>&amp;#x2212;</mo><msubsup><mi>&amp;#x03BE;</mi><mi>i</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#x2228;</mo></mrow></msubsup><mo>&amp;#x2264;</mo><msub><mi>y</mi><mi>i</mi></msub><mo>&amp;#x2212;</mo><mi>w</mi><mo>&amp;#x2219;</mo><mi>&amp;#x03D5;</mi><mo stretchy=&quot;false&quot;>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo stretchy=&quot;false&quot;>)</mo><mo>&amp;#x2212;</mo><mi>b</mi><mo>&amp;#x2264;</mo><mi>&amp;#x03F5;</mi><mo>+</mo><msubsup><mi>&amp;#x03BE;</mi><mi>i</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#x2227;</mo></mrow></msubsup></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1076\" style=\"width: 19.543em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 16.116em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.224em 1016.12em 2.672em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1077\"><span class=\"mo\" id=\"MathJax-Span-1078\" style=\"font-family: MathJax_Main;\">−</span><span class=\"mi\" id=\"MathJax-Span-1079\" style=\"font-family: MathJax_Math-italic;\">ϵ</span><span class=\"mo\" id=\"MathJax-Span-1080\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">−</span><span class=\"msubsup\" id=\"MathJax-Span-1081\" style=\"padding-left: 0.222em;\"><span style=\"display: inline-block; position: relative; width: 1.041em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.133em 1000.44em 4.396em -1000em); top: -4.014em; left: 0em;\"><span class=\"mi\" id=\"MathJax-Span-1082\" style=\"font-family: MathJax_Math-italic;\">ξ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.005em;\"></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; clip: rect(3.414em 1000.55em 4.207em -1000em); top: -4.374em; left: 0.494em;\"><span class=\"texatom\" id=\"MathJax-Span-1083\"><span class=\"mrow\" id=\"MathJax-Span-1084\"><span class=\"mo\" id=\"MathJax-Span-1085\" style=\"font-size: 70.7%; font-family: MathJax_Main;\">∨</span></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; clip: rect(3.37em 1000.32em 4.199em -1000em); top: -3.711em; left: 0.438em;\"><span class=\"mi\" id=\"MathJax-Span-1086\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">i</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span><span class=\"mo\" id=\"MathJax-Span-1087\" style=\"font-family: MathJax_Main; padding-left: 0.278em;\">≤</span><span class=\"msubsup\" id=\"MathJax-Span-1088\" style=\"padding-left: 0.278em;\"><span style=\"display: inline-block; position: relative; width: 0.809em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.395em 1000.5em 4.396em -1000em); top: -4.014em; left: 0em;\"><span class=\"mi\" id=\"MathJax-Span-1089\" style=\"font-family: MathJax_Math-italic;\">y<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.006em;\"></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; top: -3.864em; left: 0.49em;\"><span class=\"mi\" id=\"MathJax-Span-1090\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">i</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span><span class=\"mo\" id=\"MathJax-Span-1091\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">−</span><span class=\"mi\" id=\"MathJax-Span-1092\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">w</span><span class=\"mo\" id=\"MathJax-Span-1093\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">∙</span><span class=\"mi\" id=\"MathJax-Span-1094\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">ϕ</span><span class=\"mo\" id=\"MathJax-Span-1095\" style=\"font-family: MathJax_Main;\">(</span><span class=\"msubsup\" id=\"MathJax-Span-1096\"><span style=\"display: inline-block; position: relative; width: 0.891em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.395em 1000.52em 4.202em -1000em); top: -4.014em; left: 0em;\"><span class=\"mi\" id=\"MathJax-Span-1097\" style=\"font-family: MathJax_Math-italic;\">x</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; top: -3.864em; left: 0.572em;\"><span class=\"mi\" id=\"MathJax-Span-1098\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">i</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span><span class=\"mo\" id=\"MathJax-Span-1099\" style=\"font-family: MathJax_Main;\">)</span><span class=\"mo\" id=\"MathJax-Span-1100\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">−</span><span class=\"mi\" id=\"MathJax-Span-1101\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">b</span><span class=\"mo\" id=\"MathJax-Span-1102\" style=\"font-family: MathJax_Main; padding-left: 0.278em;\">≤</span><span class=\"mi\" id=\"MathJax-Span-1103\" style=\"font-family: MathJax_Math-italic; padding-left: 0.278em;\">ϵ</span><span class=\"mo\" id=\"MathJax-Span-1104\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">+</span><span class=\"msubsup\" id=\"MathJax-Span-1105\" style=\"padding-left: 0.222em;\"><span style=\"display: inline-block; position: relative; width: 1.041em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.133em 1000.44em 4.396em -1000em); top: -4.014em; left: 0em;\"><span class=\"mi\" id=\"MathJax-Span-1106\" style=\"font-family: MathJax_Math-italic;\">ξ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.005em;\"></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; clip: rect(3.414em 1000.55em 4.207em -1000em); top: -4.374em; left: 0.494em;\"><span class=\"texatom\" id=\"MathJax-Span-1107\"><span class=\"mrow\" id=\"MathJax-Span-1108\"><span class=\"mo\" id=\"MathJax-Span-1109\" style=\"font-size: 70.7%; font-family: MathJax_Main;\">∧</span></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; clip: rect(3.37em 1000.32em 4.199em -1000em); top: -3.711em; left: 0.438em;\"><span class=\"mi\" id=\"MathJax-Span-1110\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">i</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.447em; border-left: 0px solid; width: 0px; height: 1.466em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mo>−</mo><mi>ϵ</mi><mo>−</mo><msubsup><mi>ξ</mi><mi>i</mi><mrow class=\"MJX-TeXAtom-ORD\"><mo>∨</mo></mrow></msubsup><mo>≤</mo><msub><mi>y</mi><mi>i</mi></msub><mo>−</mo><mi>w</mi><mo>∙</mo><mi>ϕ</mi><mo stretchy=\"false\">(</mo><msub><mi>x</mi><mi>i</mi></msub><mo stretchy=\"false\">)</mo><mo>−</mo><mi>b</mi><mo>≤</mo><mi>ϵ</mi><mo>+</mo><msubsup><mi>ξ</mi><mi>i</mi><mrow class=\"MJX-TeXAtom-ORD\"><mo>∧</mo></mrow></msubsup></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-64\">-\\epsilon - \\xi_i^{\\lor} \\leq y_i - w \\bullet \\phi(x_i ) -b \\leq \\epsilon + \\xi_i^{\\land}</script></td>\n",
    "<td>nuSVR没有这个参数，用nu控制错误率</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>是否用对偶形式优化dual&nbsp;</td>\n",
    "<td>&nbsp;和SVC类似，可参考上一节的dual描述</td>\n",
    "<td colspan=\"2\">SVR和NuSVR没有这个参数</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>正则化参数penalty&nbsp;</td>\n",
    "<td>&nbsp;和SVC类似，可参考上一节的penalty&nbsp;描述</td>\n",
    "<td colspan=\"2\">SVR和NuSVR没有这个参数&nbsp;&nbsp;</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>核函数 kernel</td>\n",
    "<td>LinearSVR没有这个参数，LinearSVR限制了只能使用线性核函数</td>\n",
    "<td colspan=\"2\">和SVC, nuSVC类似，可参考上一节的kernel描述</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>核函数参数degree,&nbsp;gamma 和coef0</td>\n",
    "<td>LinearSVR没有这些参数，LinearSVR限制了只能使用线性核函数</td>\n",
    "<td colspan=\"2\">和SVC, nuSVC类似，可参考上一节的kernel参数描述</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>损失函数度量loss</td>\n",
    "<td>\n",
    "<p>可以选择为‘epsilon_insensitive’ 和 ‘squared_epsilon_insensitive’ ，如果选择‘epsilon_insensitive’ ，则损失度量满足<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-65-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mo>&amp;#x2212;</mo><mi>&amp;#x03F5;</mi><mo>&amp;#x2212;</mo><msubsup><mi>&amp;#x03BE;</mi><mi>i</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#x2228;</mo></mrow></msubsup><mo>&amp;#x2264;</mo><msub><mi>y</mi><mi>i</mi></msub><mo>&amp;#x2212;</mo><mi>w</mi><mo>&amp;#x2219;</mo><mi>&amp;#x03D5;</mi><mo stretchy=&quot;false&quot;>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo stretchy=&quot;false&quot;>)</mo><mo>&amp;#x2212;</mo><mi>b</mi><mo>&amp;#x2264;</mo><mi>&amp;#x03F5;</mi><mo>+</mo><msubsup><mi>&amp;#x03BE;</mi><mi>i</mi><mrow class=&quot;MJX-TeXAtom-ORD&quot;><mo>&amp;#x2227;</mo></mrow></msubsup></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1111\" style=\"width: 19.543em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 16.116em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.224em 1016.12em 2.672em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1112\"><span class=\"mo\" id=\"MathJax-Span-1113\" style=\"font-family: MathJax_Main;\">−</span><span class=\"mi\" id=\"MathJax-Span-1114\" style=\"font-family: MathJax_Math-italic;\">ϵ</span><span class=\"mo\" id=\"MathJax-Span-1115\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">−</span><span class=\"msubsup\" id=\"MathJax-Span-1116\" style=\"padding-left: 0.222em;\"><span style=\"display: inline-block; position: relative; width: 1.041em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.133em 1000.44em 4.396em -1000em); top: -4.014em; left: 0em;\"><span class=\"mi\" id=\"MathJax-Span-1117\" style=\"font-family: MathJax_Math-italic;\">ξ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.005em;\"></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; clip: rect(3.414em 1000.55em 4.207em -1000em); top: -4.374em; left: 0.494em;\"><span class=\"texatom\" id=\"MathJax-Span-1118\"><span class=\"mrow\" id=\"MathJax-Span-1119\"><span class=\"mo\" id=\"MathJax-Span-1120\" style=\"font-size: 70.7%; font-family: MathJax_Main;\">∨</span></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; clip: rect(3.37em 1000.32em 4.199em -1000em); top: -3.711em; left: 0.438em;\"><span class=\"mi\" id=\"MathJax-Span-1121\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">i</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span><span class=\"mo\" id=\"MathJax-Span-1122\" style=\"font-family: MathJax_Main; padding-left: 0.278em;\">≤</span><span class=\"msubsup\" id=\"MathJax-Span-1123\" style=\"padding-left: 0.278em;\"><span style=\"display: inline-block; position: relative; width: 0.809em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.395em 1000.5em 4.396em -1000em); top: -4.014em; left: 0em;\"><span class=\"mi\" id=\"MathJax-Span-1124\" style=\"font-family: MathJax_Math-italic;\">y<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.006em;\"></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; top: -3.864em; left: 0.49em;\"><span class=\"mi\" id=\"MathJax-Span-1125\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">i</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span><span class=\"mo\" id=\"MathJax-Span-1126\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">−</span><span class=\"mi\" id=\"MathJax-Span-1127\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">w</span><span class=\"mo\" id=\"MathJax-Span-1128\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">∙</span><span class=\"mi\" id=\"MathJax-Span-1129\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">ϕ</span><span class=\"mo\" id=\"MathJax-Span-1130\" style=\"font-family: MathJax_Main;\">(</span><span class=\"msubsup\" id=\"MathJax-Span-1131\"><span style=\"display: inline-block; position: relative; width: 0.891em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.395em 1000.52em 4.202em -1000em); top: -4.014em; left: 0em;\"><span class=\"mi\" id=\"MathJax-Span-1132\" style=\"font-family: MathJax_Math-italic;\">x</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; top: -3.864em; left: 0.572em;\"><span class=\"mi\" id=\"MathJax-Span-1133\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">i</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span><span class=\"mo\" id=\"MathJax-Span-1134\" style=\"font-family: MathJax_Main;\">)</span><span class=\"mo\" id=\"MathJax-Span-1135\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">−</span><span class=\"mi\" id=\"MathJax-Span-1136\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">b</span><span class=\"mo\" id=\"MathJax-Span-1137\" style=\"font-family: MathJax_Main; padding-left: 0.278em;\">≤</span><span class=\"mi\" id=\"MathJax-Span-1138\" style=\"font-family: MathJax_Math-italic; padding-left: 0.278em;\">ϵ</span><span class=\"mo\" id=\"MathJax-Span-1139\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">+</span><span class=\"msubsup\" id=\"MathJax-Span-1140\" style=\"padding-left: 0.222em;\"><span style=\"display: inline-block; position: relative; width: 1.041em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.133em 1000.44em 4.396em -1000em); top: -4.014em; left: 0em;\"><span class=\"mi\" id=\"MathJax-Span-1141\" style=\"font-family: MathJax_Math-italic;\">ξ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.005em;\"></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; clip: rect(3.414em 1000.55em 4.207em -1000em); top: -4.374em; left: 0.494em;\"><span class=\"texatom\" id=\"MathJax-Span-1142\"><span class=\"mrow\" id=\"MathJax-Span-1143\"><span class=\"mo\" id=\"MathJax-Span-1144\" style=\"font-size: 70.7%; font-family: MathJax_Main;\">∧</span></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; clip: rect(3.37em 1000.32em 4.199em -1000em); top: -3.711em; left: 0.438em;\"><span class=\"mi\" id=\"MathJax-Span-1145\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">i</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.447em; border-left: 0px solid; width: 0px; height: 1.466em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mo>−</mo><mi>ϵ</mi><mo>−</mo><msubsup><mi>ξ</mi><mi>i</mi><mrow class=\"MJX-TeXAtom-ORD\"><mo>∨</mo></mrow></msubsup><mo>≤</mo><msub><mi>y</mi><mi>i</mi></msub><mo>−</mo><mi>w</mi><mo>∙</mo><mi>ϕ</mi><mo stretchy=\"false\">(</mo><msub><mi>x</mi><mi>i</mi></msub><mo stretchy=\"false\">)</mo><mo>−</mo><mi>b</mi><mo>≤</mo><mi>ϵ</mi><mo>+</mo><msubsup><mi>ξ</mi><mi>i</mi><mrow class=\"MJX-TeXAtom-ORD\"><mo>∧</mo></mrow></msubsup></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-65\">-\\epsilon - \\xi_i^{\\lor} \\leq y_i - w \\bullet \\phi(x_i ) -b \\leq \\epsilon + \\xi_i^{\\land}</script>，即和第二节的损失度量一样。是默认的SVM回归的损失度量标准形式。</p>\n",
    "<p>如果选择为&nbsp;‘squared_epsilon_insensitive’ , 则损失度量满足<span class=\"MathJax_Preview\" style=\"color: inherit; display: none;\"></span><span class=\"MathJax\" id=\"MathJax-Element-66-Frame\" tabindex=\"0\" data-mathml=\"<math xmlns=&quot;http://www.w3.org/1998/Math/MathML&quot;><mo stretchy=&quot;false&quot;>(</mo><msub><mi>y</mi><mi>i</mi></msub><mo>&amp;#x2212;</mo><mi>w</mi><mo>&amp;#x2219;</mo><mi>&amp;#x03D5;</mi><mo stretchy=&quot;false&quot;>(</mo><msub><mi>x</mi><mi>i</mi></msub><mo stretchy=&quot;false&quot;>)</mo><mo>&amp;#x2212;</mo><mi>b</mi><msup><mo stretchy=&quot;false&quot;>)</mo><mn>2</mn></msup><mo>&amp;#x2264;</mo><mi>&amp;#x03F5;</mi><mo>+</mo><msub><mi>&amp;#x03BE;</mi><mi>i</mi></msub></math>\" role=\"presentation\" style=\"position: relative;\"><nobr aria-hidden=\"true\"><span class=\"math\" id=\"MathJax-Span-1146\" style=\"width: 14.938em; display: inline-block;\"><span style=\"display: inline-block; position: relative; width: 12.338em; height: 0px; font-size: 121%;\"><span style=\"position: absolute; clip: rect(1.173em 1012.34em 2.611em -1000em); top: -2.184em; left: 0em;\"><span class=\"mrow\" id=\"MathJax-Span-1147\"><span class=\"mo\" id=\"MathJax-Span-1148\" style=\"font-family: MathJax_Main;\">(</span><span class=\"msubsup\" id=\"MathJax-Span-1149\"><span style=\"display: inline-block; position: relative; width: 0.809em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.395em 1000.5em 4.396em -1000em); top: -4.014em; left: 0em;\"><span class=\"mi\" id=\"MathJax-Span-1150\" style=\"font-family: MathJax_Math-italic;\">y<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.006em;\"></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; top: -3.864em; left: 0.49em;\"><span class=\"mi\" id=\"MathJax-Span-1151\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">i</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span><span class=\"mo\" id=\"MathJax-Span-1152\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">−</span><span class=\"mi\" id=\"MathJax-Span-1153\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">w</span><span class=\"mo\" id=\"MathJax-Span-1154\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">∙</span><span class=\"mi\" id=\"MathJax-Span-1155\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">ϕ</span><span class=\"mo\" id=\"MathJax-Span-1156\" style=\"font-family: MathJax_Main;\">(</span><span class=\"msubsup\" id=\"MathJax-Span-1157\"><span style=\"display: inline-block; position: relative; width: 0.891em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.395em 1000.52em 4.202em -1000em); top: -4.014em; left: 0em;\"><span class=\"mi\" id=\"MathJax-Span-1158\" style=\"font-family: MathJax_Math-italic;\">x</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; top: -3.864em; left: 0.572em;\"><span class=\"mi\" id=\"MathJax-Span-1159\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">i</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span><span class=\"mo\" id=\"MathJax-Span-1160\" style=\"font-family: MathJax_Main;\">)</span><span class=\"mo\" id=\"MathJax-Span-1161\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">−</span><span class=\"mi\" id=\"MathJax-Span-1162\" style=\"font-family: MathJax_Math-italic; padding-left: 0.222em;\">b</span><span class=\"msubsup\" id=\"MathJax-Span-1163\"><span style=\"display: inline-block; position: relative; width: 0.818em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.087em 1000.29em 4.441em -1000em); top: -4.014em; left: 0em;\"><span class=\"mo\" id=\"MathJax-Span-1164\" style=\"font-family: MathJax_Main;\">)</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; top: -4.377em; left: 0.389em;\"><span class=\"mn\" id=\"MathJax-Span-1165\" style=\"font-size: 70.7%; font-family: MathJax_Main;\">2</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span><span class=\"mo\" id=\"MathJax-Span-1166\" style=\"font-family: MathJax_Main; padding-left: 0.278em;\">≤</span><span class=\"mi\" id=\"MathJax-Span-1167\" style=\"font-family: MathJax_Math-italic; padding-left: 0.278em;\">ϵ</span><span class=\"mo\" id=\"MathJax-Span-1168\" style=\"font-family: MathJax_Main; padding-left: 0.222em;\">+</span><span class=\"msubsup\" id=\"MathJax-Span-1169\" style=\"padding-left: 0.222em;\"><span style=\"display: inline-block; position: relative; width: 0.757em; height: 0px;\"><span style=\"position: absolute; clip: rect(3.133em 1000.44em 4.396em -1000em); top: -4.014em; left: 0em;\"><span class=\"mi\" id=\"MathJax-Span-1170\" style=\"font-family: MathJax_Math-italic;\">ξ<span style=\"display: inline-block; overflow: hidden; height: 1px; width: 0.005em;\"></span></span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span><span style=\"position: absolute; top: -3.864em; left: 0.438em;\"><span class=\"mi\" id=\"MathJax-Span-1171\" style=\"font-size: 70.7%; font-family: MathJax_Math-italic;\">i</span><span style=\"display: inline-block; width: 0px; height: 4.014em;\"></span></span></span></span></span><span style=\"display: inline-block; width: 0px; height: 2.184em;\"></span></span></span><span style=\"display: inline-block; overflow: hidden; vertical-align: -0.374em; border-left: 0px solid; width: 0px; height: 1.454em;\"></span></span></nobr><span class=\"MJX_Assistive_MathML\" role=\"presentation\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><mo stretchy=\"false\">(</mo><msub><mi>y</mi><mi>i</mi></msub><mo>−</mo><mi>w</mi><mo>∙</mo><mi>ϕ</mi><mo stretchy=\"false\">(</mo><msub><mi>x</mi><mi>i</mi></msub><mo stretchy=\"false\">)</mo><mo>−</mo><mi>b</mi><msup><mo stretchy=\"false\">)</mo><mn>2</mn></msup><mo>≤</mo><mi>ϵ</mi><mo>+</mo><msub><mi>ξ</mi><mi>i</mi></msub></math></span></span><script type=\"math/tex\" id=\"MathJax-Element-66\"> (y_i - w \\bullet \\phi(x_i ) -b)^2 \\leq \\epsilon + \\xi_i</script>，此时可见会少一个松弛系数。其优化过程我们在SVM原理系列里没有讲，但是目标函数优化过程是完全相似的。</p>\n",
    "<p>一般用默认的‘epsilon_insensitive’就足够了。</p>\n",
    "</td>\n",
    "<td colspan=\"2\">SVR和NuSVR没有这个参数</td>\n",
    "</tr>\n",
    "<tr>\n",
    "<td>缓存大小cache_size</td>\n",
    "<td>\n",
    "<p>LinearSVC计算量不大，因此不需要这个参数</p>\n",
    "</td>\n",
    "<td colspan=\"2\">在大样本的时候，缓存大小会影响训练速度，因此如果机器内存大，和SVC，nuSVC一样，推荐用500MB甚至1000MB。默认是200，即200MB.</td>\n",
    "</tr>\n",
    "</tbody>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SVM算法库调参要点\n",
    "1. 一般推荐在做训练之前对数据进行归一化，当然测试集中的数据也需要归一化。。\n",
    "2. 在特征数非常多的情况下，或者样本数远小于特征数的时候，使用线性核，效果已经很好，并且只需要选择惩罚系数C即可。\n",
    "3. 在选择核函数时，如果线性拟合不好，一般推荐使用默认的高斯核'rbf'。这时我们主要对惩罚系数C和核函数参数γ进行调参，通过交叉验证选择合适的惩罚系数C和核函数参数γ。\n",
    "4. 理论上高斯核不会比线性核差，但是这是建立在要花费更多的时间来调参上。所以实际上能用线性核解决问题我们尽量使用线性核。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 案例\n",
    "## 不同分类方法的区别"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "d:\\env\\pythonve\\test36\\lib\\site-packages\\sklearn\\svm\\base.py:931: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  \"the number of iterations.\", ConvergenceWarning)\n",
      "d:\\env\\pythonve\\test36\\lib\\site-packages\\sklearn\\svm\\base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 792x792 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZkAAAEFCAYAAAAxAZr2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzsnXd4VUXawH9z7r25Jfemh5ACafQuXQQVEaUqYl2KqNh11RV1dd21oH6ua8eyKnZFWRBUREVEQKr0HlpIQkjvPbntzPfHuYSEGiSBAOf3PHlyz8yZmfecM+95p7wzR0gp0dHR0dHRaQqUMy2Ajo6Ojs65i25kdHR0dHSaDN3I6Ojo6Og0GbqR0dHR0dFpMnQjo6Ojo6PTZOhGRkdHR0enyTivjYwQokIIkXCc+DQhxOUNzOsWIcSKhuZ9uhBCXCqEyDjTcsDJ3c9zDSHEICHE7jMtx+lCCBEuhNgthLD4jpcKIW73/R4vhFh4ZiXUEEJ8KoR4vhnIUU9PhRBrhRCdz6RMjcVpMTJCiIFCiFVCiFIhRJEQYqUQoo8Q4kIhRKUQwnGUNJuEEPf7fvsJIZ4RQuz1nZ8mhPhYCBF3KnJJKe1SyhRfGY1a2ermrXP+cCxDKqVcLqVsf4Zk8hNCvCqEyPA1flKFEK/74n4RQkw9SpqrhRA5Qgij77ivEOInIUSJT4fXCiFuPU6xjwOfSClrDo+QUs6QUl7RWNd3jvIKcMRzORtpciMjhAgA5gNvASFANPAs4JRSrgYygGsPS9MF6AR87Qv6BrgKGAcEAt2BDcCQppb/bOLgC+Fsy1un6fA9tyeA3kBfwAEMBjb5TvkUmCiEEIclnQjMkFJ6hBAXAouB34E2QChwDzD8GGWagUnAl416MafIWaYf84DBQojIRs739COlbNI/tMpdcpz4fwCLDwv7DzDX9/tyoBpo1cDybgV+qHOcDMyqc3wA6OH7LdGU5k7ADbiAioPpgTTgEWArUAr8D7Aco9xbgBV1jiXQxvf7U+Ad4EegHFgDJNY5twPwK1AE7AZuqBM3Eu2FUOaT/Zk6cXG+ciYD6cCyo8h1KZBR5/gBIAmI8R2PAjYDJcAqoFudc9OAv/uu3wkYT3RPGpDf5U1d587k37Gu8SjP4VTu4+PAPl9dSgKuOawergRe99Wn59EaeQ8dQ16rr/yL64QFAzVAd9/xCuCdk7gHFwPJh4UtBW4/jq7cDewFin26IurE3wbs9MX9AsTWiXvTpxdlaA3PQXXinkFroH7pi7/9KLJ+Cjzv++0AlgDTAAGY0XoU6UAu8B5grfs8ffqRA3xRJ2wKkAdkA7fWKeuE+R0m26/ApDNdp09ZJ06D0gUAhcBnaC2f4MPiW6G94Fv7jhXfgxrjO/438PtJlJfgU0wFiAT2A5l14ooBpU7lrmsInj8srzRgLRCF1gvbCdx9jHKPpjh18y5Ca0kagRnATF+cv09JbvXF9QQKgM51Kl9X3/V081XOg/cmzlfO5758rEeRq7byAv8CNgLhvuOePmXoBxjQWp9pgLnO9W/2PSPrie5JA/PTjcyp38frfekU4EagEoisUw89wF999ckK/BPtxXavry6Jw2SbDnxY5/guYLPvtw3wAoNP4h7cB/x4WNhSjm9k5gNBQGsgHxjmixuD1lDs6LuefwKr6qSdgNazMqK93HPwGWs0I+P25aFwdP34FM0Qh/qex/N14t5A61GEoBmgH4AX6zxPD/ASmvGw1gmbCpiAEUAVvndeA/I73MhMA14703X6VP+afLhMSlkGDESrSNOBfCHEPCFEhC/+AFo3fIIvyRDAgtbqB+3hZ59EeSloLbwewCVoLZ9MIUQH3/FyKaV6EpcwTUqZJaUsQqsUPU4ibV3mSinXSik9aEbmYD6jgDQp5SdSSo+UciMwB7jOdz1LpZTbpJSqlHIr2hDiJYfl/YyUslJKWX2MsoUQ4jXgSrSXRb4v/A7gfSnlGimlV0r5GVqPpf9h13/gsLyPdU8akp/OIf7UfZRSzvalU6WU/0PrAfStk2+WlPItX32qBl5EexmOB9aj6cOkOud/BlwvhLD6jm/2hYHWq1E4CR1EMxblJ3E+wL+llCVSynS03sTBe3EX2ot4p093/g/oIYSIBZBSfimlLPRd66toL/y6c1+rpZTf+e7VsfQjCu0dNFtK+U/QFAbtOfxNSlkkpSz3lX1TnXQq8LSU0lknbzcwVUrpllL+hDYy0r6B+R1OOdq9PKs5LRP/vgpyi5QyBuiC9lDfqHPKZ2gVG7Sx4K+klG7fcSFaj+Rk+B2tZXCx7/dStBfzJb7jkyGnzu8qwH6S6U+UTyzQzzehWiKEKEF7GbQEEEL0E0IsEULkCyFK0YYVwg7L+8AJyg5CGxJ8UUpZWic8FphyWNmt0J7P8fI+3rWcKD+dQ/yp+yiEuFkIsblOXBfq14l6z8xnqN6RUl6EVhdeAD4WQnT0xa9A6z1c7fOI7AN85UtejPYyPRkdLEZrqZ8Mx7sXb9a51iK0oaxoACHEFCHETp9TUQnanO0x78UxGInWE3mvTlg4Wi9uQ52yF/jCD5Ivj3RsKPQZw8OvpSH5HY4DbVTmrOa0uzBLKXehdVG71AmeC0QLIQYDY9GGfw6yCOgrhIg5iWIOGplBvt+/c2Ijc6a2oz6ANhwYVOfPLqW8xxf/FVoXu5WUMhBNEQ6fpD2R7MVoPaZPhBAXHVb2C4eVbZNSfl3nnJO5Lw3JT+fEHPM++lrw04H7gVApZRCwnfp14pjPTEpZLaV8B61OdKoT9TlaQ28isFBKmes7vwpYzWHOOSdgK9DuJM4/HgeAuw67F1Yp5SohxCC0OZEb0IakgtDmlxp0L+owHe2F/5MQwt8XVoA2F9y5TrmBUsq6jcyT0Y2G5Hc4HYEtJ1FGs+R0eJd18LU2YnzHrYC/AH8cPEdKWYk2QfcJsF9Kub5O3CK0CbBvhRC9hBBGIYRDCHG3EOK2YxT7O5oHjVVKmQEsB4ahDb1tOkaaXLQ5m9PNfKCdEGKiEMLk++tzsJWJ1popklLWCCH6onnYnTRSyqVoPaRvhRD9fMHTgbt9vSUhhPAXQow8mkt5A2ns/M5WTEIIS52/k/U8Ot599Ed7ueUD+NyIuxwnL4QQDwltHYbVpz+T0OpVXV34HM3J5g4ODZUd5DHgFiHEo0KIUF+e3YUQM49R5FogSAgRfVJXfXTeA54QvjUjQohAIcT1vjgH2hxIPmAUQjyFNgf8Z7gfzelmvhDC6htSnw68LoRo4Ss7Wghx5Z/J/GTz83no9UJ7953VnI6eTDnaBOYaIUQlmnHZjjZJV5fP0LrGn3Mk1wE/oXnglPrS90br5RyBlHIP2ljoct9xGZACrJRSeo8h50dAJ19X9rsGX90p4hubvQJtbDYLbdjg4GQiaJO1U4UQ5cBTwKxTKOtXNAeDeUKIXj5jfgfwNlrLNhltUvbP5t+o+Z3F/ITWaj3498zJJD7efZRSJgGvovUuctEm8leeIMtqX5octBb1fcC1ss46LillGpoXmz9az7muPKuAy3x/KUKIIuAD33UeTX4X2mjFhKPFnwxSym/R9GGmEKIMTfcPuk7/AvwM7EFz8KmhYcNjRytHog0pHwC+F9oi0r+j3fs/fGUvov58z8lyMvldBSyVUmadQnnNAqHdWx0dHZ3GQwgRjtbIu+A4E+46x0AIsQaYLKXcfqZlOVV0I6Ojo6Oj02Sc13uX6ejo6Og0LbqR0dHR0dFpMnQjo6Ojo6PTZDTJhnFW/xAZEHwyy1p0Thdh5WnaD5PfGZWjLkmFhQVSyuMtSjur0fXh/MNiM2HP3XvSenYu6kKTGJmA4BjG3zfvxCfqnFYmrZgMhGNs2epMi1KPLh9/tv9My9CU6PpwfvGk8joZy7YjOvTHEH5ym12ci7qgb99+nvCk8joZ0OwMjI7OuYIxMJTxP47R9ewwdCNzHjBpxWS94uvoNCEPjyqi8PHJCMCg61k9dCNzjvPwqCIKV+gGRkenqZi0YrKuY8dBNzLnMHrl19FpOsaMCiLwcW3fUF3Hjo1uZM5RtEl+vfLr6DQFk1ZMhhUgDIaTntw/39CNzDmIbmB0dJqGDj1j6DdN2zhZ16+GoRuZcwzdwOjoNA1PKq+TMW27Prl/kuhG5hzi2R5LSNHnYHR0Gh3dQ/PPoxuZc4RJKybrBkZHp5F5tscSUt7+Uu+9nAK6kTkH0IfIdHQaH73h1jjoG2Se5egGRkencXl4VJGuV42I3pM5i9EVQUencTm4tkx3TW48dCNzlqLvRaaj03jorslNh25kzkJ0TxcdncajdmEl+uR+U6AbmbMMfYhMR6dxOLhrMuj61JToRuYU8XiceNw1mC0BCCGatCzdwOg0Z6SqUlNdgsUahFCat0/RQV0CXZ+aGt3InAJrFr/DmiVvIRQjoRHtGXPz+9jsYU1Slm5gdJozGSl/8MNX9+P1uDAY/Rg97m1iEvqfabGOiq5Lp5cTGhkhhBm4Foire76UcmrTidX8Sd29hM1r/scFg2fgZw5h/673+eWbx7nmlg8bvSxdKZoHui4cHWdNGfNm3Etit38QHN6H4vx1zJtxL5MfXYrZEnCmxatF772cGRrSp/0euBrwAJV1/s5rstM3EdryMsyWMIRQiIy7lpyMLY1ejm5gmhW6LhyF4oJUzNYWBIf3ASA4vA9mSzjFBWlnVrA61NUjXZdOLw0ZLouRUg5rcknOMhxBkVTuWI2UXoQwUFa0DXtAy0YtQzcwzQ5dF46C3RFBdWU2zup8zNZwnNX5VFflYHe0ONOi6b2XZkBDjMwqIURXKeW2JpemGbNryzyW//wfXM5y4jsM4bLRT7Fry09sW3UPFmsE5cU7uOaWjxqtPN3ANEt0XQAK85JZMOsRigv2ERyWyLAbXqH/4PtYt+xeAoI7U1a8g/6D78ce2LiNrpNF16HmwTGNjBBiGyB959wqhEgBnGju5FJK2e30iHjmydy/niXznqNdz2cx2yJJS3qbJfOf59rbPuXAvlU4a8qJiu2NPaBxWm66cjQvdF04hNtVzZyPbiYi7ibiujxHQfYS5nw8iVsfXkRs24EU5acQEv4o4ZGdzpiMeu+leXG8nsyo0yZFM8DlrOC3758hI+UPbPYwBo/+F1GxvQDYv3c54THDcAR3BiC2w13s+ON+FMVAbNtBjSqHbmCaJeeVLkgp2bTqUzav/gKAHhdO5IIBtyCEoCg/GcXoT2Ts1QBExl5NfsYPFOUnExHd9YwaF9D1pzlyTCMjpdwPIIT4Qko5sW6cEOILYOJRE56l/Py/KVRUQLueL1FZtpfvPruD8fd/T2BIKyzWIGqqDo2QVFccaBKvGV1Bmifnmy7s2DCb9cs/I7HbEwCsX/4ifhY7XXpdj9kaiKu6CI+7EqPJH4+7Emd1IWZr4BmV+UnldTKWbQd0/WluNGROpnPdAyGEAejVNOKcGVTVS9qexfS78kcUgxmrPYbSgjWk71tJ15Cb6NzrOrau/ZrdG5/CbGlJQdavDLvh5dr0HreTFb+8TPq+1djsYVwy4nHCIzuelAy6gTkrOOd1AWDPtl+IaXMrjqAOAMS0uZU9WxfQpdf1BIW0pl23kSSteZCAsD6UFayjfbdRBIW0rk2ftHEum1Z/CcAFF06gU8+xTSqvvs1S8+Z4czJPAP8ArEKIsoPBgAv44DTIdtoQQkExmHHVFGLxj0JKiaumAD8/fwDMFgfj7p3Lzs3f46op59IRX9Ai6tD7ZuGcxynMLyC6zf1UlO3lmw8nMOGB+TgCIxtUvm5gmjfnky4A+JltuGoKao9dNQX4mf1rj4dc/Sz7khZSlJ9MSL9LSOx0RW3c7q3zWbbgFeI7PwzAsgWvYDD60b5b4484PjyqiMLHHwV03WnOHG+47EXgRSHEi1LKJ06jTKcdIQQDhv6N9cseJTxmBFXlyShKFbFtBvHNhxPJz92Lyc/KFWNfoHXigHppVdXL3u0/0veK7zEYbTiCO1FZsp39e5fTpfcNJyxbNzDNn/NJFwD6Db6H2dMn4HIWAlCQuYDr7/iSVYteY9vaOQB07XstAy5/+Ii0SRu/o1Xb22vXzHja3k7Sxu8b3cjoW/KfPRyvJ9PT93N2nd+1SCk3NplUZ4BeAycTHBZPRsof+MdfSNe+4/j6v9fhdhtI6PwwFaV7+PbTyUy4/3tCI9rVphNCQSgGPO5KDEYbAB53BQaD6YRl6gbm7OB804XwyE7cdM9sdm76DoDLr5rNri3zWL/8ExK7/A2A9ctfBwQDLv9bvbQGox9ez6H1qR5PBQbjiXXhZND15uzieHMyr/r+W4DewBa0IYJuwBpgYNOKdnpI2jiH33/6N6qq0iKqA9ffPgMA1eOhKH8P/YZ+j9HPQXCLflSU7GTDio+44tqXACgpTGPt0vcJCk1g26r7iGkzgaryZDyuHBI7DT1uuc1JUdLLyvho8zYqnE4ujY9ldJvEo4adx5wXulBRmsPM927A6azEbPbnprtn1a512b7+WxI6P0B49BAApPSyff0XtUbG465hzZJ3qakuJefAe7icJSiKkey0WY22fux0TO7XeDxM37yVlMJiEkKDuaOH5p1+eJjFqG/72FCOua2MlHKwlHIwsB/oKaXsLaXsBVwAJJ8uAZuSvTsW8OvcJwmPGUVsh7vJy0rm8zdH1MYLBFJ6a49V1Vu7u2xZSSZf//c6yiushESOQTGYqChaQkzraP5y7xz8zPZjltucDEx2RQUTvv8RS0oxnTPdvLtqPe+s33hE2BfbdpxpUc8Y54MueF0uPnltKEZLK+I63ofR0opPXhuK1+UCDi4IOqQL2k4Xvt+qytxPJ5O6dxv+QUMIbtGb4tyFOOwVXDf5c6JaH9H5O2kmrZhMxrLtTbotjCol9y1YxI6k/fTI9rAjaT/3/vwr9y34tV7YfQsWoUrZJDKcizTEHHeou8JZSrldCNGjCWU6baxY8BLhMVfQut0tANgc8Wxb9QAVZblsXPkJFlsYW1bcTVzHuygv2UVZ0Vb63Kz1YnZtmUdwi0G0ajupNu2+Lc8ycNhjxy2zORkYgB/3pXKhauMvSigAcdKPJ5N2crG01wv797YdTOza+XhZnQ+cs7qwdf1XCGGkY6+pCMVIWOQlrF10HZvXamtl7AHh7Nv2OqrXhRAG0na+y4DLHwCgMD+ZovxULrj4i9q0m36fyAUXTSKsztDyn+HZHktIeVvzVGtqnUkpKWF/UQnvy9YYFMFA6eC2ojQMCKYTWxt2Z1E6KSUltAkOblJ5zhUaYmR2CiE+BL5EW/U8AdjZpFKdJrxeN4riV3usGPyQUvLVO2MIDB9Ey7gbyNj7BWk7P0AxWPB3hNZu5S+9XoRiOiyt94gy6tLcDAyAV6qYOPQdHBMKUnJEmN5yA85hXfB4NOOBMGgBwoBAYfuGOYCDoPBLsFe6ObD3U0zmMBAQHd8XAKl6URRjvbSKwYhUj68PJ2LSismkrDh9+uKVEqMQKL6qrgAGITBSP8wkBF5dHxpMQ4zMrcA9wIO+42XAf5tMolMkbc/vHEhZg78jnK59bsTkZ6uNq6kqYdv6mTirK7DagjAYzOSmz8dqb43FFklq0rsgPTiCexHf6T4AHEGd2LPpeXpe+gm71j/Ovp2/0rHHGNp1G8GGFddh8Y/BYo0kI/nj43qTnU4DszIjk3XZOYTarFzbri0207EnXq+Mj2PC1h1EqSYiMDFDKWZ0m0R+Tk6pF3ZNh1NrkZ4jnFW6UFyQyq7N3wPQocfVBIfF18ZJVWXn5m8pzEvGYguiOC8Fr6ea5K0vEx49hPzM3/B6qqgsy6fn4GkoiomIVsPZsGQ87Xv+k9KCzaxf9iGjx79DaIu2+NtDSN3xOiEtL6Eo53f87SGEtmj7p+RuTNfk/aVl/LgvBYCRiQnEBh57EXViUBBBdhvvludzkWpnpVJBuN0fAbxbcSgsyG4jMSjolOQ6nzihkZFS1gCv+/6aNZtWfcbapR8QFjWMquSlJG38lpvuno3RZKamupQZ74zBYu+A2dKSrLTPMJvDCQjtQea+mUjpwWqPxe3Mw2By1OZp9HOgqtq4tNHkwOtxAhAclsC1kz9n9aJplJStpkf/6+g54NajynU6DcyM7Ul8unErl6l2litu5u9O5vOrRmI2Go56flxgIB+MuJL31m9indPFmIROTOjSiWvatT0i7HznbNKF/OydzP5wPKGRmgPKptXXcv3tMwiP7IiUkgXfPEb2gV0EhvWnIGsWUvUgDFYqSnZTnLcWk18gJrMDo9GKENprQigmDEYrqteF0c+Bq0LTBcVg5NrJn7H85/9QkDmTsIi2DBr+GYrh5CfHD7omN4au7C4qYvL8BVyqavOj47cn8dGoYbQPCTnq+UZF4f0RV/DGmvXMLSohMSSMD/r1BqgX9n6/3hib+Zc/mxPHc2GeJaW8oc7mgPVobpsCSilZ8cvLdBv4Plb/GKSU7Fr3CPuSFtK++2h2bJiNxb89bbtryxy8XifZad9SXZlBYtcp2Byx7Nv2CsFhCeQd+BH/wHZYrJGk7HiLwNALyEqbS2HOSkqL2pOfvZPwyI5ERHdhzKQj1+KNGaW1cr6bX3JaDYyUkjc3bOQNWhGlaEN//6rM5rf0dEYkxB8zXaewUKYNu/yEYecrZ5suAKxZ8l+i4scRlXA9AH6WcNYs+S+jxk2jpDCN1N1LueDSGRgMFgJCu7Nj9RSEYsDmuIDoxBvJTf+BkvxVSOkiffd0QiMHk5/5K1L14KopIi3pbeLbDWTfzkUkdBiCxRrI0LEv/Gl5O/SMod+0K4HG05UPN27lejWIqxVt7iRMNfLhxq28fPmlx0wTaDbz9MUXHRF+tDCdhnG8psbBIYGzYnNAKVVUrxOzJRzQFlj6WcJxuTSffVdNBX6+uOStr1CQtZSWrUdSXrKL1B3TUFFQBASEXonZ5iR953+x+gfj7+9PTU0KmclrCQztTkZ6Nlv+GM/wG18jvv2l9WQwBoYyapAXIQRSytM+B6NKiUtVCVOMtfcgDCNVbvdpKf8c5qzSBdA2fLXU2RXcbG1BTdlOX1wlfuYgDAYLxXnr2bXhScKiLgWgMHsZxXmrkdJDaMuBgCQ3fT7lRX9gtgVisweQtvN1jCY7FZVWFs97kfS9Kxl81dN/WtYnldfJmLa90RdWVrpchNV5xYVjZK/PW07n9HG8Ff/Zvp9DgOVSyr2nR6Q/h6IYaN32EvZtf41WbW+hsnQvxXl/0CrxCfYlLSI3cyu56etxBHcnL+MXug98H/+ABKRU2bL8LirLU4jt/CCKYiAwtCfZqTO56Ip7UBQTa5a+jcEUQYfeWkstKLw/S3/8v3pGRuu9eAkPODhhLthZXsnWsnLCXXBxqxgUIdhXXMLm/HyC/PzwSEmV202fyJbEOBxHXNPJYlAUBkZG8W5ePjfJEFJkDeuVSqZENmx7G52jc7bpAkDbLlew+rf3sdi0l3bG3k+4cMhdlBTuZ+uaL3HVFJKxbya5++cT02YirdpOAOCAPZbMlFmERFxEULg2VGS2RWO3VXDh5Q+yetFrJCctptdlMzGa7HjcE9n0+0R6DryNwJBjN6ZqqkpI3b0EgPj2g7HYgnAbBZEzRvA+EBzekrzqChLUPHpGNM4nM4YkxPFxwSZaSm1O8iulmNsSLmiUvHUaTkMGTeOACUKIWGADsBxN0TY3pWB/hvbdRvDbd89QlLMKIRTCItqyefUXbPljBiERFyIUM7s3PotUPVjt2oZ+QihY7a1xO0tI2/kOASHdcLtK8Xqq2LJmBplpGzFbIwht2aW2HJu9Nc7qktrjg8NjhwwM/PrzPKZu3cnFAwexZ88evk3dz4hW0Ty3YjW9hT+7PVVUodLN4M+rrOONKy6jd8tT/8jTvy+7mOeXr+bJnGxCLBamXTSEVgGnbsB0gLNIF2LbDGLZTy+xY80jACgCrLYQPn9jOI6QLvjZWpOx9wuEMGCzx9ams9ljUYSB0vy1VJXuweIfTXlxEjUhMXz+xnCsjniMpkCMJm2ew2iyY7aGUlNdQiBHNzLlJVnM+eh6unXV5vVmLn6Fz978P+687U7iVTNuKdnu3Ukfo4P3cDK2c3vu7XXqxmBs+7aUu1y8vEPrwd3UuTNj2/85ZwSdP4+QDXTFE0JYgTuAR4BoKeXRZ5KBiJhucvx98xpHwpPgv8/3ol3P/8MR1AFV9bDjj79SUZJClwFv1oZtXnY7zpo8QiMGERTeE4+7grSdH9Ch1zNI6eXAnk/pNvA9Ni+7DWdVDl0GTEP1utiz8Vk69HkBiy2StKS3CAl1MPzG145qYAAG9WrDh9M/oFu3brjdbq4bM4b05H08QxTthAWPlDziTecm31qUWf6VzL1+zBHX5PJ4+GrnLoqdTkYmJNAu5Nzzze/y8WcbpJS9z7QcDeVkdAHOjD4smP0YFRVWWre/DYD03R+Rl/ET4dHDie1wOwD7d00nM+UbzNYIYhJvAiAz5X9Y7XG06fo3Ni69mR4Xf0j2/u/JOzCfiFajiWkzno1LJ9Gq7c2ERQ2mIHsJOalfcuuU3+p5ctZl8fd/56JeLZjy8EMAvPbKKyz44kv6VJuZ4Kv/X3gLKMbDJCWMe0hnzrVjiLT7H5HXov372ZybT4+IcC6PjT0i/mznbNOFhnBCFwkhxD+FED8DC4E2aIoV09SCnSyq6sVZXYo9oA0AimLE4h+HqrrqhfkHtEH1VFGY8zu56T+RmfwVNkccwS36Yw9sh8tZhKIYcQR3QlXd2APaEBjajdiOd7Hjj0dY/9v1BAZbGTLmuWMaGK/XS0lJMZ06aS03k8lEhw4dqPB6SMQMgFEIEoSZYjy0EWZyqio5nCqPhytnfsOX67aydts+/vLdD3yffE4sMD8rOVt0AaCyPA//wENu5/6B7VFVFbtv+34Ae1BHwIjbWUh22lyy0+bichaR0Pl+TOYgzNYWuJxFOII6AgJ7UAcMRiud+71EVsos1i68mtK8BYy97bNjGhiAmsp8utVZyNu1e3eqvCptfLoA0FZYKMZLoDDikAqFNdVH5PPgwt948rdlbN6RypO/LePBhb+d2k3SOS00ZLhsLOABfgR+B/7wuXKecaqriinI2Y1sd4scAAAgAElEQVTNHkZoizZERPcgbfd0gsP74nFXUpSzHIPJRtqu6cR1vJOq8lSK81ahGCx07PMiQWEX4PU62briLopyV1Gcu5KAkG5UliVTWrAWP3Mg6Xs+o3X7W/APSABUht/wOp0GTqyd4A87bCRKSsmeXTto174jb7/zDn+9/3727NnDksWLaR8YxMzyIm4ihP24WCMrGSGC+EotxCNVluxPJ9Jup8LtokNICC+sXkOYW/BvQxwGIViilvHv5X/QJTQUtyqpcLuI9Pcns6KSUKul1ne/pKaGPcUl9cJ0GoVmqwtej4vcLG1fr4ioLrRK7M/2DbNRDNqLPDP5SwxGA+l7PiUwtDsAGckzMBoNhEYNIbGL5tuQsv0tDuz9gqDwXricRfiZg0nf/T7+jjAykmcQGNodP0sYBpM/rdsOYuwtHx9XrrKSTCy2lnw8/QP69esHwMcffEDb0GDm5ZbQRVoBmKsW0Vf4s1ItJ191syAlFavBSGFNNa0cDoqdTlZkZPKeIY5wYSJfurk7I43v9ibTs0ULcqoqaWnzp8ipPY7OoaGYDAbcXi87CgvrhemcXho0XCaEcKBtAjgQuAHIlVIec1PA0zE8kLV/A99/fidWeyuqKjLp2GM0rRL689P/HsJsbYmzOh8AP0sYzuocVG8NQjHRtc9NbF3zJQNGLtJWOAO7Nz5HQdYSTH6BuF0l+JkdDBnzHMGhcXzz8S24akoQipFufccx7YM3gCN7L6D1YB7/251s2biWwMBADhxIp7KyCrvdzhMJ0fQMDeHRjUlsKyrEhEAi8QIWFEIwUIQXBUGMyUyhohLmb6NbMUwwaLsMFEg3d3vTMAkFBYhQ/EhXnSQkJpBfWMjw1jEMjY7mgV8XEy38yPY6Gd42gb9f2A8hjpS3uXA2DRGcrC5A0+tDTXUps6dPwFmjvWDNFgujx73NzPdvRFW15+52lWKxRVJTlVW77is0vB1maxAB4aMJbam56BbmrGDv5hcRwojHXY6iGOjc6wYuHvkkM98dS2H+ntq04+7/HqPR7ygSaSRtmMXqRS/RLiaSXampOF0uBDC2Q3se69mDl1avZU5yMqqU2FEoR8WKggWBSSiUSg8JJhuZqovL4luzJjmdD4yHXPHv8KTiFVAhvcSZrKR4qgkOCSbA4cDorOHViy7i4UW/UV2preex+lv4cOQwAszHlvlMczbpQkM5YU9GCNEFGARcgrYD7QG0Cc8zyo8zHyK+yyOERAzA465g++r72L3tR9pd8K/asK0r7iGhy1+pLEslI/lzJj20gPKSTJI2ziUrdS7RCddTXZlJcf5a4jrdh8dVgEI2Y2/9uPalfN9TG3HVVGD0szH2Km0R19EMDMD872aRdSCVhb8swGw289XXXzP323l8OWchEf97moxl25kxZhRvrNvAt9t28brSikfUdG4T4UQpZkwSnlezGO8NIdvr4puqchZJNyNlEEEYmKeW0E3YmKK05CFvOr1UKxdLOxtNZn5dvJhrRo/m5517uN8bRj/FTiVeHknez6VxsfSP0j3MTpXmqgurFr6O0RxPu17a911Str/GD1/dT2BYXxK6aGH7tr6CYrTQvvXTbF52OyNuepMWUZ349tPbqKycW+tJlp32LVZ7LLEdJrN383PcOmUxNn9tHvDmh37G4/EtTD6OcQGoLM9nza/PM3fOXOLj40lNTWXs1Vfx7eiRRPhrcy3/GjSASd27MuabuTymRLFaLceFyjWKpmffyiKsHgMPKuE8vC8VD5LNahU9FBubVc1x5j0Ry8cUUOL28E8RyUsVhXw7fz6vv/IKDy9eQmwF3Cu1Ec13yvN5Z/1GnriofyM/AZ3j0ZDhspfQhgamAeuklGd80YWqeqkoyyK4hdb9NprsOIK6kJP+U72wgJBulBRsJP/ALwhh5pNXhyClRKo15KbPJ2PvF6iqC6s9nvTdHxAW0YExk6Yf0er3s9iPOf9Sl/S0FC66aABmszZEcdngwbz55jSEEOTdNBWWjcWbc4CH+vRiR0Eht2SnogIfygJsXoVCPERjIgsXfYU/X3mKcSqSW7wpGBDEYeZfhij8hYF2wsI8WYIKGPaV43A46NW7N3O++47eBk2J/YWBzsLC/rIy3cg0Ds1OFwCKCtIICr+itt4Ghfdn/871xMb8pTYsOGIAOfu/I2X7mxhMNn6Z/Rhe1YVisOLvCGDtwqsBsAd2oLryAPu2vsBV49+tNTAHOZFxOcjFv9/LirAw4uO1nkd8fDwxkVHkVVXXGhmA1gEOHuzbm6lrN2ACIvDjaTUTAAcGAqWBaOFHpMlCtUkwtTITgwoGBP9QonAoRvqrdqbJHKbKLKxuI9nZ2Vw2dCi///gT16rBCEW7B31UG4tLSv/8jdb5U5xw4l9KOVJK+R8p5armolRuVyWBwbHkZSwEwFVTSGnhBgxGG3kHDoUV568jP3MRrTvcQUzbcVhskfS9Yi42RxxR8TdywaWfcsHFn6B6ihk9/l3G3fct/o7wemWpqpcBvSuoqqqoNTAul4stm9aRn5dLeVkpuTlZqKpKu45dWLRoEZmZmeTk5DBr9mxi4xOprKwgO/MAOZNuxa2qAEwffgW/j7sRK4JJShjvGuN43xBHLm7yVBeL1FIQECKNvCviCMHAlTgIFkaKpIckWc0zhmhuFCFIl4u8vDxWLF9OpMXKb2op+dJNlupik6yi3TF2i/WqKtkVlVS53ZS7XORWVuobYR6H5qgLHo+ToNAYCrMXIVUPUvVQmL0IRRHkZfxcG5af8QtedxVGv0C6XPg6Euh20Tu0bD0SkzmQ3pfNpPdlMzFbg+nc8xruemINrRIvrFeWlJKqigKqKg59mllVVQpz91CYuwePx0mLGJW//H4rrWxWMnPzWL9+PVlZWaxbt46M7GzsJiMVLhdbcvOo8C2MnNSlM6tuHoe/2UykMPGhIZ4PDfFEYKQID1u8lWR7nORXVvG8iGYAdrpgobOw4pWS32QZQ0UgLxtaUa16KCgsZM7s2YTarCwR5RSqbgpVN0uVCjqEHX1LGSklhdXVFFZX4/J6ya6owOU9tQ0+dTQa7MJ8MjTVGLSUkpULX2Xjyo8RGFClB4PRhsddQWzbQaTtXozZ2gIhjLichUgJUq3B4h+Nx12B6qkhtuNdBIb2IGnt40jpweOuIK7dxYy5efoR5QWFV/Lh81dRU1VBZVUlDz3yFLHxCUy5/1aEELhcLqQEh8NBRGQUb33wFROvGUpebg5+vjkXPz8zVW4XUkr8EBiNBt7v152u7TvjVVV6fPoF3xnaogBfqoXMlcUYta93UI2kC1aSqcEPBQ8SI+BE8hcllOuUEAqkm7u8aSgGA15V21HZK1WMCGpQGRgdzdtXHrk9TGppKff9/CsVThdVXg9SCGyKgQi7P+8OH0oL27G9hRqbc3Ecui5NpQ/79y7jx5kPIaUBr6cKfDtnmy0OAoJjKS8rwVWTj5Qq0utCYsA/II6q8hSk9BIc3pfEbo+wZ9NzlBfvBCRmi4ObH/wZq399hxGP28mvcx7kQMpqAFolXMigEc/x7UdjqSgvwOv1YjAYCAgIwl1Rxgud2/JTUTkL96XiJxRcUmtcGXy7YRgQuJHc06sHd3TXduW5/YefGVpopJ9iZ6NayUtqNgKtvruRhAgjHikxIajEiwSMCNoLC/9QorAIhTs8qRQaJYpXxSYUalQvB01FkJ+ZOdePIchsrndtTo+XR39bytrsHDxSRQD+BiMeJC8NuYSLoqMb/dkdi3NRF86qXd5Sdv5K0qYf6TX4a8y2lrRuN5kuF75OtwFvk3NgGzZ7C2I73EXHvv9Hj0EfYrEEYzDaiIy7jn5XfM8Fl35G5r6vUb01RMVfh58lnG4D3iY7fSvZBzbVK2vMqCC+fPUmJt96M3+sXsWCn37i4/ff4NEHJvPIlCm8/J//EB0dzdIli1my+DcGXzKI28ZdhTevkM8M8fzPkMhoEUxbt4H3lVgCMTBViWaYDODBddtw52UigWirjTWygjWykpWygk8N8cwyJDJKBGNFIQ83HxnimWFIYJQIQkXbHmOU0F4Cq9QKjAjChZFPlDgipJFbRBgzjW34wBDPjtx81mZlH3Evp/y6hJE1/twrwwnDyMdKHJ8TR48KA08tOePTDDonoKa6lB+/foi2PZ6hRasROIK70e2id+l20btY7YlI1YU9IIGuA96i20XvENxyAIoBDEYrfYd+S78r5qGqbrJSvyGu471IJJ36vITVnsjGlUd+yXL9sneICPGwbs1q1q1ZTUSIh3lf3EinDvEsWbwYu93Oxx99xK8LF/DO9Ok8sTOZX/elMtUQzUxDIlMN0VgQfCBi6YqN0SKIqYZo3tuwmZSSEqSUxAUH8YdSRZnq4RU1h38ZophpbMNzhmj8EJRLD/8wRPKZMYFnDNFIwInKZBGGRSikSyeFeDB5VZ5RorhCBtAZGzMNicw0JBKvmvh481YOb1h/sGkLVbml/JdW+El4WoniU+J4XEbw999+p9TpPE1P9dzkrPqGaG7mdkIiLsFoslNZlkL3QWNrPcQCQ3vRKr4129e/g9kaQXVlNp16jmXzqo+JjLsKAIutJQEh3Ula+w8MRjOd+r2Mzd6a4PC+5GUlEdlKW2U8ZlQQHo+HfXuSGPeXvwAQHR3NwIEXMWfOHMaNG8e0t94iJjqay4YMQVVVunXrRnbmAcYKB4G+XWtHKIE85E2nhTDRS9hIwckIAvjOVcSFPy/Bq6qoUvIq1fghGCmCatOOVAL53lvMZSKgXn4LvKUI4BZvCgEYKMRD3+goYrKrsQsDaTgZrmgGqIUw0clr5o4FC+kZ3oLXhg4m2GLBo6okl5XysqEtX6uFDKoj8zAZwMNFmafnger8aUoK0zDbWhAY2p3MfV/TMnY0Noe2ODE8ZgTlhT/hceWwe8M/ALA7QjCZbLSMvRrFoM2rRLQexb5tr5CdOofELg8RENoFt3sE2b4h57oU5W7j4ftvqp1vHHfT9TzxjycZP348OTk5BAUG8vCUKZSUlBAUFITF345/lYcuQusRdxE2wjGRL7wMUwL5WZYyQYQRjJEx3/2ATSi4vF4MCNZSjj9KvbShGHGh1guLxEQ0Jh5WDxChmsjBTbuQEGoqqumi2viGYoYrgZiE1pYeqjp4bccu5ien8Orlg7nAt33N9rx8hqoO8oWXcEz1yghTTKSXldM1vH7vR6fhHLMnI4T4QQgx71h/p1PIgwQEx1BeshWJ5ppcWqD1PryeakoKNuEIjOLWKYsZdt1Uxt83l8Gj/onBYKl3XnnxDlRvNQmdH8Bmb62FlSQRGKx5oByc4I8MMREREcnq1drwQFVVFVu2bMVsNrN69WoqKyrYtXs3vy5cyLatW+nQoQNGPz82UY3H11LarFYRgYkaqbJL1hCBic2yilbR0QSZLLyvxDFHacMlSiBeq5mNsrJeWiOCTaJ+fpGYeFW0wqIY6BoXzffXXcPQ+Fh2KE4kEIqRrbIKgBqpkiKd/FOJomWRm6eWrgC0Lc3DzRa2yipaCBPb5KEytsgqov2P/eno85HmqAv2gEiqK7OpqcrGbG1JScF6zalFSkry1uBxV3PT3bO4euJbXD3xLW66exY2/2BK8tceOi9/DVJVCWl5MeHRlyOlpKxwA0GhR24P4whrw4qVq2rTrly1CilVli1bhr+/P1nZ2Tw3dSrbt23jualTKS4upkB1keubusqVbnJxEyoNbJKVRGAkV7qp9FNwmC08LFswx9CGKSICp1FQiKde2kI8lOE9IuwupQX9hB1rkI13rhzCW1dcRq5XKzcCI5tkZa3MG2UllwoHd7tDeOCXRZT7eigxAQ42i2pCpYFc3PVl9rpo6X/6ho7PRY45JyOEuOR4CaWUvx8rrqnGoFWvh3lf3k1edjJer4qrJh9HcGdqKjMxGG30H3wb3fuPr5dmw/IPWbHwVewBbamuzECqHsz+LamuOEBQaFeqqzJIaH8JQ8f+H9eM1ibID07wr1m1jCl/vZXOnTqTmppK/4GD+Wbm59gUAzWqF6vRD2Hxw+l0Emz1J7+sBIsw4I9CsFRIx02iLZCs6jK8UhKBkSyDSo3Xo83PIHAiMSOoQMWKwB8DoRhJw0lE61YUZmVj80KIMJKm1tASE6V46dwynPeGax5FHlXloYWLSckvxCwF6Z4a2hqsZHud9BF2xokQ/qNmk0QNYWYL/xx4IXY/Ew//uoREYWGXuxIrCsEYyRRu3hs+lJ4tIxr9+R2L5j4OfSq6AE2nD5tWf87qRW/iZ2lJZVkqVv8YhGLE7SwhrGU8N9wxo975Rfn7mPH2mNrdyGuqcvAPTKCqLBWbPQbFYEJRXNx450ys/toE+cFGV0lxIQ/fPhq7TWvRV9U4sdn82bN9Cx63BwkEBwVRWFZKaEAgJSUlCCEwSIgTZvZLJ4F+ZiyqINtTTSvMZCkePEi8qpdADJTgxYZCFdq8iAlBPGZScWI0m/GzWqgqKSPBYCXFW4UNBX8MVPkJ5lx7NaFWbWHn1zt28u76TbRW/NjjriZS8cPrc7h5XonhW1nMD7IEo6JwY4d23Nq9K7fNX4Cpxkuh10mF6iUWM+m4GNelIw/1PX1Vs7nrwp/hrJr4B+2LftkHNvHtp7fRuuNfMZnsGE0O0pLeZPDox0jseOQkd15WEslJvxAY3IrAkFjcrkocQVGUFWdgs4dxyejhdIyuxM8oCDys0ZKfl0PS9i2EhrWgc9ce9GsbwVAcTJDBpODkeWM+Ma1a0Ta1gImEkixreNaQhzQaeeH55wgICMBut/PAgw9SWlDA5Uog/VQbb6g5TDXEEIGJt9VcJDBSBPIvNQOPovD3xx4jMTERf39/HnrgAbw11fS/+BJGjBxJaVkZ/3nhBT4dOoTEYO0loErJlrx8KlwuIu3+zNy5m81703lJRvMvmUkiFiYooaTg5AWRw2ejRxBg9uP7vfv4fNNWbiEUCworRAWOmBBeGnJpkzy/o3EuKlZdmlIfigtS+O37Z3C5HIRFXQZASf46AoPgyuv+fcT5NVUlbFs3E4CYhP5UVxbiCIyiqlJbFR8d2xujyVJrXOBQo6umppqN69cA0LN3P267cRQVO3bylNQ2dn3ekEdM/55k/LGRf3q1oainDXnUBDsYOGggo0drX0r48ssvWbVyJSFOyT9lBI+rB5gkwhikOFiulvOZLODfSiumqpnkKir9LuzPnXfeCcCML79k8+pVWIOC+duj2tczP3rvPYYHBjCx06Etc9JKS0krLSPS7k9ycQnPLl/JW7RmNZUsV8t50qDtTv2iyGH0BZ24oWN7Vmdm8dSylYzxBBApTGRLN/P9KvjxhrHH/bpsY3Iu6kJDFmO2BV4EOgGWg+FSyoQmlOvY8igKUbG9GHbDqyyY/SjB4b2pKk+lRVQ7EtpfdtQ0LaI60SLqyC87hkW08ylTJX6ylKmP/Y21q5cTGhbOE0+/RL8BFxPeoiWXXKYpkdfrpcrrZpISglEIOmKln8HBwuS9BAs7k2UqQRhoJ0wkXjWWZ559Fq/Xi9FopH27dqzPz+cOGcb/ZCFDRSBxQmsV3qKEMcWbzmOGSAZIB1W9O/Lee+9xQc+eZKam0j0okN9Sinjl1Vcx+Sr7hj/+YGNuXq2RUYSoHWMGeKxfH+4sLOKx4iz2eKqZaoiplbmv8Gdjbh7Xd2hHjdfDUAK4TAkEoK208Gh2VqM9r3OJ5qYLoH2hddh1/2HmezeQ4y4CwFWdyYgbZh/1fIstiD6X3H3UuA49Y+gQVYGUkhkfvc782ZoDwI0TJnP7PX/DYrEyYOClteebEYxUgwjxfb/oek8AH23YRCe3YIpMB6CraqUiIorflizml4W/aF6WJhMdYuO5cncxlUIlECNDDFr9G2II5DtPCVVCZZISznRLGenp6bz99tv4KQr79+4hMTyciY/9nSuGal/99PPzY+arrzCxzrXEBQYSF6jl2T4khMzScp7YnoTBK7lNCSfENwc5Vg1iWXomN3ftTLi/jRCMXGs45Oa8TK0itbSMzmGhJ/NYdOrQEO+yT9C+Y+4BBgOfA180pVANIbHj5Yy7dy49+g9n6DVPMXrc24iT/CRq3QWWj//tTgL8/Zj3/Xc8OuVvPHz/LaSl7qt3vqIo+CkG0tH8+71SckB4sKJgQ+ENQ2tuVsLY5a1ixdJFDBs2nIW//MIrL7/Mnr17sVuspOMiQBjYL521Xi77pYsADHilJE06iYuL49HHHiNp/XoebpfIywMH4LBaSfZtjun1ekneu5cgy7EnI/0MBqaPHMb9l/TDZjDWlxl3bdoQi4V0xV1PliC/5rvtxhmmWeqCPbAlEx+cT79Lx9Pv0vFMfHA+9oCGD3eOGRXEmFFBdIjS1oItmf8py36ZxccffcjHH33Iz/NmMeurT45I5/J6SeWQ51Wa4qLa42GPdPKsIZpnDdHskU62J20lPCycWf/7H7NnzSIsLAxpUNivuGudVyqk5mhcIb0U4iEAA6myBsVo5IsvvmDHtm1cosCsEcNoYbGwZ9eu2nJ379xJ8Al6Gnf27M6bwy8nKjSI/XVlFk6CbVp7IdhsoUB11ZOlQHURfBw90zkxDfEus0opfxNCCCnlfuAZIcRy4M9/Cq+RCA6LJzjs2J8VPh51DYzX62XlssVs3bIFs9lMy5YtuWzwYNb9sYK4+EQAVq1YyrRXnsOpennGnM+F+LPLVUEJBmpQuUdpgUkohAkTfaQ/y3OzeebppzCbzezbtw8/j0q1y81j3nT6Cwc7qOZxbwaRwsRyWU4XrDzsTacAD/v27OHXn37ipYsuZEC01q1/ondPbpkwgSuHDWNXUhL+VZUMbn38L26aFIXBsa3510X9eWbVGi7ETpriIjjEUZt2TNs2fLUticerMmiJiT+o5JUBl/6pe3oe0Gx1wWwJoH23hn+48+BXXIEjNnpdtvgXHvjrX2nbVvv2ygN//Suz587jxvHaZwNysjN54elHScvcT6rFSbYswu12s16twCwN3KyE0trXS79ZCeUtTx4PPqDll52djVUxsHvnTnZ6VXZSSSAG7vfup4/wZ62sJAgDH6n5rJeVBIlAbpkwgRs7tOcvHbXhsLs6d2LSJ5+wb4+2j9ofK1fy2RUn/lR41/Bwnr90EDfP+5EM1QPADkMNn/fSptyiHXauTIjngeQ0LsDGJqq4sk08UXbdEeZUaIiRqRFCKMBeIcT9QCbQOJ+uOwPU/UTyQcVSFAWr1UZWVhbx8fFIKTlwIIMBgwMA2LljK489MJnnn3+Oxx7bzQv/+Q9Z2dlcGhDAtGnTKK2sIA8P0fghpSRHulEUhaysLKqrq3nw7nu41xlEhAhmuqmI/S1DiM72MtRrp1qoGCX8TjmXE4BJCFpVlvPUFUPq7aA8MjGB+MBANqal0iM0mKG9emBsYM9tVJtEEoKC2Jibx6VWC0PjYmvTppeVU1xTQzfs+CForZhZmZ7BoJhmuYP9meas14VDcy3eo85BAtgdAaQfOFB7nH7gAHaHpgsup5M7Jl7DiOFXIrxOOnbsSIsIrdcUsm4di36YTzaHNkPIlm68QpL+/+ydd3gUVdfAf2d303tPIIEAIXSlI4og0hUVKQqKgl1R7FhefbG8rxXxVVEsoCCoKH5iAQtFpfcmvbcESAjpPdnd+/0xs2EDaZAO83uePMzcmVt2mDPn3nPPPTcujvz8fO68dRRdE3O5n4YsM2fxp5eNnKx87lfBZIudKOXKDJIIVy4Mwo+DJhOPtGlZbEFklK8Pc68fyF9HNZPck9cPJFif9C+PKF8fvh92U1Helxs3KspbYLOx4WQCrXHHBxOtxZ0NJxMosNlwNaI3XzAVUTKPA57Ao8B/gGuBMdXZqOrCzfM4f38+mj/eS6PvjUO4e9yT7Ni2hY8/eJPQsDCGj7iFUaNGsmfPHgptij79rgPg7yW/M3zYMAb0709ycjKvvPoqN998M7/88gt5KWmYESbY4hgofuwljxMmG9f2vY477hxDVGRDeue70d2k9YYeswfxfHo6Wd5uzElLxqSEVKx0xItEsw03Py8mXtm96KXediqJqRs2k1lQQO8mjbnrsraYz9MsCNA6OIjWJdiV/zp2jH7Kl7F6pOcEVcC/Dh3luSuNIIIlUC9lwXnUAtroPTUlmbdfeJYDu3cS06oNT7/yFgCT35zIgf17WPb3Ig4ePIi7uzu//fY7M75dAMCB/XtA2Xn8scfYv38/t91+OwMGDCA/P5+//1iIxWLh68JkEmyFCPCnyiC0UWM+mzaN7du3k5uUzGjRRtGjlStr7aeJ7dKdL9evxc2uRamIwIVQswvLTDnMvGZQ0bxjal4ek9ds4GBKKs0CA3iqexcC3N05X4I9PLilZYtz0g+mpWHPt/K0WWufUopH8uM5lJZOy6CSw9EYlE+5SkYptQFA78E9qpTKrPZWVQPtWyfy6KgBjMn3JhwXZk/9iONxx1i85FeenTCByMhI3p40iY1btjHo+qHcPOJ2XPWFZ9nZWZxO0FbN3zZqFNbCQl5/4w3MNjv3E0y4yY+pptOsDHalQ6fueCQnM+D6Idw+9n4+m/ouaU46IRVtFJWelsYDphDCcWGaSuJoqC9jgr0Z0jiySMEcTEtj3B+LudMeSDiezN62l+yCQh7r2qnKnouHxUKanPkApWLD3WL02kqivsmCYyL/7FFLYWEh9wwfRGx8KqNt7qw6tox7dg5CPNzo2rkjL//7BX797TeW/PkXw24dw5yflhAZFQ2Ai4srycmnyc/Pp3nz5nw7Zw6Db7gBU6GVfpYAetg9WWhKZ6VbIZd1aM+tMTEkp2Xz2IR/89XMz8hVdgqUHVcxUYgiszCfbZs30NPkTS98WKYyWaoyiGziz5wO1xLpo5kbCu127vt1IbFZwmi8WZWVyn0pC5lz8w24XECnqyTcLRayla1Y+7KVIQ+VpSLeZZ3RJjx99B/oUtAAACAASURBVPN04G6l1KZqbluVMWSwP19+Oo3ehe70172oHiuw8NwvPzDs9lGMGDECgMnvvMOYsXcx8o57iuU3m8ysWLGCiRMnEhkVxcyZMzGZTAywetFf94p5UYXzfPopLBYXUlIz6N1vEJ6eXjRv0ZoR/a9kamYyYVZhgVseviENuTJTFbXlWRXBk0lxPPnEcOKX7yiqd/Hho/S2e59ps7Lw8v4DVapkborR5mSmFp4iTFlYYMrgiU5dqqz8i4n6IgtnTGJZJUYNP7h/D1kJidxnC0VEaG1z54ETJ5EgX/794ouICJ06dWLFipVc229QkYIBsFo1U/Bdd99N3z59WPLnn7i7u+Nmy+N+e6BWHu7ca40nqlEj5i9YwGdfziMyKppnX3yNU0eO8PLqNXTNt7DezUqjZrHE7dzJQ6aQorzrVTbNfLyJyE4DXckcTE0jKyeP+4jS7lPuPJQTx6G0NFoEVs0oI9rXl04R4bx88iRd7Z6sN+XQKSKcxr6+VVL+pUpFugBfAOOUUtFKqWjgYTRBqxcUCZw1m2zbGVtxHnbNJTknpygtJycHs+VcvZuenkaPHj0IDAwkMTGRcePGaQEyLaZi5SkF0c3b8tX//YGnpxbOPDAomO/+WEnj+++i8LabeGPaV0Q0jCJHVLG8gr4dAGBL0OzhLmYTeWfdZ5aqDTcX6OHOnJtvoFHbRhTEBvN6n17c2DymSuu4iKizstCyY2SRl5iIEOIrpW5LYTKZyMrLKQocaQOyC3IpKCjEatUmxK1WK/n5+efKgwh2u52BAwZwLC6OgQMGaJ6JFkux8gqVnXybmZlzFtCmXXs9q/D2p7MY8q8Xyb7leob860XuHv+0tiDTOS8Kj4dfQAFWhyyYTBQq+zl1WKpQHkSEt/r04sYubcmKCeDGLm15u0+vOr3hX32gInMymUqpooiJSqmVIlKnzATpqfFsXD6N/LxMYlr3Ibbd9edM8JsE1rjkE2hPIcxu5jtzOjaziV9//ZUAf3+ioqL45NPPGHvvI+eUf+jgPvbt3k6jRo1oERvLxx9/jFKKFZKNl1XRQFz5wS2HR559mdvG3n9O/uCQUMY//WLRua+vH3cs74OfmIjAha9VMj0HaZ5BrlPnUTBuKACDmzVl9radzLaeJgwXfjClMbb95VX+/II9PHikU4cqL/cipM7JwtmjloL8fD7/dAp7du+hZauW3PPA+CKzrwOlFHYXM29JMlcUuLDaJR+rTfB0cWHcw48wcOAAFi1aTLPYVjRtFlssb9zRQ9jtdlatXk3//v1ZtGiRtvbFx4tXko9zjfiw1s1K68u68J+3pmA6y5RlsViKWQqsViuuvj68mnmCXsqbZWTh6utDz2v6Yek7iIJxQ7EmxNFIKWKCA3nrdAJX2D1Za8ohJjiQJv5+Vfo8LSYTI1u1LP9GgwpTESWzXkQ+BeYACrgVWCoiHQGUUpursX3lkpWewJypQwmKGICbZzP+XvAmMdG5DB58b7GenJe3D/0HX4+riytLNmzEkmfmpUcfZd369Xz73Vy69+jFYxNeYuDgm0uoRdHn2mvJyspi/YYN3D56NO+//wEDbhiKKijkcG4uTw0eUkrecwkJDcPT35+94eEcMplQJwvpfMVVRdcje7YlfvkOwsKj+PqmwczctoNDefk80aQFA5temMu2QZVQJ2ShpIl80PZ2eeCu2zhyLB/foB5s276U9WvX8PlX3xf72IsIvqEhdBtyMxtWr2H7zu2Mf/IJAD786CMKbNCx8xXcff+j5ygJZQcXFxfatWvHqpUradeuHVu2bKFBs+Y06dOMvSmpdGnTjrsfevycvCVhMplo3u5yEuKOsNzbm7QsD5pHRRfldZ06D4CCcUP53+UtmX0qjb3JqXQOiuCuy9piMkYZdZ6KKJn2+r9nrwW4Ek3QSl5mX0Ps3voTfiHdadzyXgC8/WL5+vPXeOCB+4rdd92Nwxl1cx/uu/de5v70I8uWLiUkJIThw4dz3/0PcHXvAaUqiSbNYlnw43c8/thjNOrWjbcnTcJkNvGft6aU2bbJb7zE6r8X4x8SwmvvTCU8oiFL/pjPjGlT6N+vL6+//joAO3bsYPxjjzPqTq3NRbtoJp2gYUgDXjC2i60r1KoslOd+fOjgPnbt3EXbq2ZjMlkIbtiXHavu4PCh/TSLOeNN1ax5S8LCG3I0Po4Cs/DCiy8ycuRIAHx9fVmxZiMPjp9QYhuiopuQn5/Pgf37ueGGG5g/fz75+fk899JbtGrdrtS2L1k4n2kfTgbgvkeeou+AG9i/dxczp3/E/r07WbF8OS4uLhQWFnJtn77ntNl16jwYN5R7IgKxdKj60bxB9VER77LeNdGQC8VmK8RsOuPGaDZ7YLcVnHNfVKNoZsxZwGcfvoPNZsPdyfXRy8uTwsJz8zhoEBFJjx49mPv99xQUFNCiRQtyc8veY+KhO4ezc8UybhJ/9h+M58aeHbnjoUf5Y/4PREVF4um0Ba2XlxeFBcU3WnSMZgzqDrUlCyXFESsJa2EhZrNr0fYXImbMZlcKC4q/22azmU9mfM9H773J+g0bi72Lnl5eZcqCSYSAgECOxcXxyquvEhwcTEBAIGWNJ3758TteeuIhhuhbUEx4YAz3PfkcX8/8hAH9++Pq6opFn/uxWCy4ubmd02Y4Y0q2JsRhCS97IbJB3aEi3mVhwOtAA6XUIBFpDXRXSp27s1Et0LztIDavvAUPnya4e4Rz8vDn3DxiVMn3xrZi0gef4+rmxmOPP85DDz7Izl27WLt2HU//e1KpdfS77kZmz/yY5555hsjISCa9M5khw28r9X673c6q5X/ymTmaUHFBKcVz9hNM//g9/lyyhNzcXEaOGkXLFi2Kyrtx6MhiZRSNZhLiMBsCVSeoSVk443587or80mjWvCVBwb4c2/shAaG9SD21jKBgX5o1P3eOwcvbh2defI2Ona/gjVeexUvfCfXttyfx/EtvlVmHj58/7du357pBg/jt99/ZsGlLiXU4mPrmK4w1BXGDSYty7o+Zrz/5gPFPP8lto0YxbPhwXnnlFQYPHsxvv/+Oh5d3qeW5Tp1H6LcTizpghrKp+1TENWMmsBBooJ/vQ1uUVicIa96N96bPxcdtI9bMbxh9xzCeeObFMvO89Np7xLbpyJuTJrN63Sa+mDOfkNDwUu9vFtOCqZ/P5deFS5j8vw+4dsCNjH/qhVLvt1qt2FH44uhRCoFiwWazERgYSExMDNOnTeOjqVOZ+PKrpZbnOnUeCrAlGQEr6wgzqWZZODuOWIhvxRQMaHMls779icvbuJKfMpPL27gy69ufioKqlkTfgTfw9L/+y/QZs5g+YxZP/+u/9B14Q5l1TJv1Ixk5hbz+1iQycgqZNuvHMuuwFhQQ4NSfDcACNhshwcG4uLgwc8YMDh46xKOPP16h8k6NfLVorsbhfWZQd6nInEywUmquiDwPoJSyijit3qtFNDOCjRDfLvTq8UOF87m6ujLgupsIDgkjODiUxtHNys1zeYfOfDLj/ypcfoPQCN5NPsVtKoBDKp/1KpOu3Xvw7HPP8fDDD3P48GGysrP57psFNIxsVGpZhtmsTlEtslBWHLHzJSAwiEnvf3xeebpc0YOMjPSi44rU8dqkqRUuv8eA65n+3TcEKu1zM12dJrZDNya/+7+ikDQnTpzkiWdePmdEXxbOoxoxmzGHNCg/k0GNU5GRTLaIBKFNbCIiVwDp1dqqcrD4BRVbD3C+/Prz9zw4dgTHDuzgm5lTGXf3LUXrA6qKOb8t53SjMJ7nBF+6ZzNx0od8OH0O3v6hPDL+Mb77vx/5+Ivvy1QwwJm1M8Zopi5QpbLgGLUMvlqbyD+fUUtVEXfsCMMH92TLur/Zsu5vhg/uSdyxI1Vax8tvvk+3wTfymukUr5lO0W3wjcz4dgFj7hvPSy+/yksvv8qY+8afl4JxcGrkqzQbPxplsxmjmjpKuZuW6e6ZU4C2wA4gBBiulNpWWp7q3KTJYasuLbhfeSiluKpjM2Z/OZPWrVtjs9m45daR3PXgE/QbeGPVN7gKiNo9j4NTvrpo7c/1ZaOmC5EFKC4Ppbkf1xb/evohoiPDGP+Itj5syocfciQ+kdffOb/RUF3Asb6sPstJfZGF86Ei3mWb9e1nWwAC7FVKFZaTrVrQzGMlh8qoKHa7ncyM9KIw5mazmZiYGFJTUqqolVVPXKuhwFeGE0AtUxlZKC2OWG2TmnKaAX16Fp23aNGCrdt21WKLLhzXqfOKOmQChqzUEUo1l4lIFxEJB832DHQCXgMmi0iNhyR13v+lMpjNZjp37c7kd98lLy+PTZs389dff9Ghc7eqaGa1YTgB1B6VlYUAP3Oxify6omAArrjyGj799DOSkpI4deoUn376Gd2u7FXbzbpg4loNJWzSpGIhaQxql7LmZD4FbTtFEekJvIm2E2A68Fn1N+0MVaVgHEz64At27t5P+w4dGP/oY7zy5gc0j21VJWVXJ5E926JsdcLn4lKjUrJgMde+Waw07rj7ITp27cG1ffvSp18/OnbtwR13P1TbzaoUqV7NcJ06j8iebbEmxBkds1qm1DkZEflHKXW5fvwRkKSUelk/36qUal9iRqpuTqakDcaqEqVUvQt+VzBu6EXnSVPX7dCVkQWAtpd1UN/PX1rt7awMju9AfZOH8gjIPkjihAn1xnxW12XhQihrJGMWEcecTR/gL6drFXF9rhQOr5vq9LipjwLl8KQxqFFqVRZqAhGpl/JQHo5RjWE+qz3KUjJzgGUi8jOQC6wAEJEYqtmFuarNYxcTmhOAITA1TK3JgkHV4Dp1Hs3Gj9bMZ4bs1CilKhml1GvAU2irnHuoM3Y1EzC+uhpkKJjycax2NmzNNUNtyYJB1RLXaqgxqqkFylyMqZRaq5T6USmV7ZS2r7pCmhsKpuIYTgA1S03LgkH14RySxuioVT9Vu83iBeLY1e9CV/BfihiRAAwMLhyH+UzZbIb5rJqpdSXjHAywpkNq1HcMJwADgwvHMJ/VDLWqZAzzWOUwnAAMDCpPMfOZIUtVTq0pGUPBVA2GE4CBQeVxnTrPiBRQTdSKkjEUTNViOAEYGFQex5oaMBRNVVKjSsYxwe8Ia25QNRhOAAYGVYehaKqWGlMyzhP8dSlA4MVC2KRJxmjGwKCKcI59ZiibylEjSsYwj1U/qV7a7p6GQBgYVA3GNs9VQ7UrGUPB1ByGE4CBQdVjKJrKUW1KxrFFMhgKpiYxnAAMDKoe59hnhrI5P6pFybh7uhTbt9yg5jCcAAwMqgfH4k0wRjXnQ7UoGQ8XuzHBX4sYoxkDg+rD2BDt/KgWJWMxV0epBhXFGM0YGFQvp0a+WuTRaUQJKJtSd8asVKEiScDRKi/Y4GKlsVIqpLYbUV0Y8mBwHlx0slAtSsbAwMDAwADqQBRmAwMDA4OLF0PJGBgYGBhUG4aSMTAwMDCoNi5JJSMiL4jIThHZJiJbRaRbFZd/jYgsqGh6FdQ3RERaO50vFZHOVV2PwcWHIQsG1Y2lthtQ04hId2Aw0FEplS8iwYBrLTersgwBFgC7arshBvUHQxYMaoJLcSQTAZxWSuUDKKVOK6VOAIhIJxFZJiKbRGShiETo6UtF5D0RWS0iO0Skq57eVU/bov/boqKNEBEvEflCRDbo+W/S08eKyDwR+UNE9ovI20557hGRfXp7ponIhyJyJXAjMEnviTbTbx8hIuv1+6+uigdncNFhyIJB9aOUuqT+AG9gK7APmAr00tNdgNVAiH5+K/CFfrwUmKYf9wR26Me+gEU/7gv8oB9fAywooe6idOB1YLR+7K+3xwsYCxwC/AB3tPUVUUAD4AgQqLd1BfChnn8mMNypnqXAZP34OmBJbT9346/u/RmyYPzVxN8lZy5TSmWJSCfgaqA38J2IPAdsBNoCi0UEwAycdMo6R8+/XER8RcQf8AG+FJHmgEJ74StKf+BGEXlaP3cHGunHfyql0gFEZBfQGAgGlimlUvT074HYMsqfp/+7CYg+j3YZXCIYsmBQE1xySgZAKWVD6+EsFZHtwBi0F3CnUqp7adlKOP8P8LdS6mYRidbLrCgCDFNK7S2WqE285jsl2dD+n8430qijDEd+A4NzMGTBoLq55OZkRKSF3tty0B5tGL4XCNEnQxERFxFp43TfrXp6DyBd7135Acf162PPsykLgfGidxVFpEM5968HeolIgIhYgGFO1zLRepIGBhXGkAWDmuCSUzJodugvRWSXiGwDWgMvK6UKgOHAWyLyD5qt+kqnfKkishr4BLhHT3sbeENEVqGZFM6H/6CZFLaJyA79vFSUUsfRbNfrgCVo3jPp+uVvgQn6pGmzUoowMDgbQxYMqh0jdlkFEJGlwNNKqY213A5v3Y5uAX5Em4z9sTbbZHBpYciCwflyKY5k6jMvi8hWYAdwGPiplttjYFBbGLJQTzBGMgYGBgYG1YYxkjEwMDAwqDYMJWNgYGBgUG0YSsbAwMDAoNowlIyBgYGBQbVhKBkDAwMDg2rDUDIlICJZItK0jOtHRKRvTbapIoi2R0d8bbcD6u4zqg7qyvsiIi+LyFfVXc/5ICK3i8iiCt5bZvtFxE1fOBpeyvWxIrLyQttaV9AjTw8s43q9kvNaVTIi0kMPC54uIikiskpEuohIdxHJFpFzwkPoK3kf0Y9d9Rdzv37/EdFChkdXpl1KKW+l1CG9jpki8t8LLUt/8W36hyhDRP4RkcFO16NFROnXs0QkUUSmioiL0z1HRCTX6Z4sEWlQmd9YH7kU3peLDaXU10qp/lVU3P3AcqVUQhWVVyuISG8R2S4iaSKSLCI/ikhDp1veBF6rrfZVNbWmZETEF21zoSloIbsbAq8A+UqpNUA8xWMSISJt0UJfzNGT/g9t/4jb0GInXY4W3K9PDfyE82GNUsobLYz5VOBb0SLXOuOv39MO6A48fNb1G/SPmePvRHU1Vl9FXae4xN4Xg5J5AJhd05VWgzzsAgYopfzRti3YD3zsuKiUWg/4SjXv6FlTcl6bI5lYAKXUHKWUTSmVq5RapJTapl//ErjzrDx3Ar8qpZL1IVo/4Cal1AallFUpla6U+kgp9fnZlYnIXSIy3+n8gIjMdTqPE5H2+rESkRgRuR+4HXhGHz3MdyqyvWhb1qaLyHci4l7eD1ZK2dGExAtoXso9p4DFaB/HSiEij+rmhUj9fLBomzml6SOCy5zuPSIiz4oWwypbRCx62tOl/c6yyqsGLon3RR/5rhKRKfq9e0Skj9P1BiLyiz6SOyAi95VSzq8iMv6stG0iMsSpzQ/qo7pUEflIpChApUlEXhSRoyJySkRmiYiffs0x8r5Lfwapejld9PLTROTDs37PSqfz9/V8GaJtiFahTcREpBHQDC1emSMtSH8WGSKyXr/unKeliCzWn9VeEbnlrLzz9bwbROS/Z7VTicjDIrIfTQmUV56biLwjIsdEs0Z8IiIeJf0WpVTiWZ1EGxBz1m1Lgesr+GzqtpzX1kY2aJscJaN9HAYBAWddjwIKgUb6uQmttzpEP38TbU+JitbXFEjTy4lAizZ73OlaKmDSzxUQox/PBP57VllH0CLBNkDrVe8GHiyl3rHASv3YjDZCKQBC9bRovT7Hhk8NgH+Au8+qr28FfuM1QLx+/G9gM2c2nuoInAK66e0Yo5fr5lTHVv25e5T3OytYXrltNt6XEt8XK/AEWtDIW9GCPwbq15ehjYbd0aImJwF99GsvA1/px7cA65zKvVx/fq5ObV6ANrpupJczUL92N3BA/53eaPuxzD7rff1Eb0N/IA8trEso2gjzFGc2QBuL/v7r56OBILSQ+08BCYD72e0v4blcj7b9gHPat8BctE5bW7Qo0A5Z8wLigLv0ujoCp4E2Tnm/BTzROnRxZ7VToXX2AgGPCpT3HvCLfr8PMB94o4z3qxHa+2VHe2/HnnX9SWDexSDntTaSUUplAD3Q/jOnAUl6ryRMvx6HJlCj9Sx90F7qX/XzIIpvpFRefYfQwoC3B3qhhRc/LiIt9fMVShtpVJQPlFInlLZx0ny93NK4QkTS0ITxHbRdAE+ddc9p/Z7jQDaaaceZn/SeRJqIlBWnSUTkXWAA0FsplaSn3wd8qpRap7SRwJdo+2xccdZvilNK5Vbgd1akvCrjEntfTgHvKaUKlVLfoYXev15EotCewbNKqTyl1FZgOnBHCWX8DDSXM6H87wC+U1qEZQdvKqXSlFLHgL+d2nQ78K5S6pBSKgt4Hhgpxc0r/9HbsAjtfZ2jlDqltAjJK4ASw/Urpb5SSiUrbSQ5GXADKrJVsz/a/wcAImJGM49OVEplK6V2oHVAHAwGjiilZuh1bQZ+AIY75X1JKZWjlNp1Vl4HbyilUnR5KKs8QZOHJ/T7M9GiRI8s7ccopY4pzVwWDLwI7Dnrlkz9N5dGvZHzWp34V0rtVkqNVUpFovVEGqD1CBw4m0DuAL5RShXq58loPczzYRlaL6CnfrwU7YPRSz8/H5wnH3PQenylsVZ/oQLQejslmQiC9Xs8gVXAH2ddH6KU8tf/hpRRlz/aBOkbSt9RUKcx8JSTokrjzFa2DuJKKK+031mR8qqUS+h9Oa70bqLOUbTf2gBwfMScrzlPGgOglMpH6+WPFhETMIpz5zNKa1MDvVznOixAmFNaotNxbgnnJf4+EXlKRHbrZpk0tLmx4JLuPYtUiu8TE6K3yfmddW5zY6DbWe/n7UB4KXlLeved08orzxPY5HTtDz29TPSP+pfAz2cpcR+0kU5p1Bs5rzMuzEqpPWimhrZOyfOAhiLSGxgKzHK6tgTo6rBDVhDHR+Nq/XgZ5X80qiyCqN4rHAfcIaVszKT3LmYC3UWkIsJ3Nqlova4ZInKVU3oc8JqTovJXSnkqpeY43XM+v7Ui5VUbF/n70lDvHTtoBJzQ/wKluBddI85sFnY2X6J9CPsAOUpzkKgIJ9A+Ls51WCmuSM4bff7lWTRTXoDeqUqnYjtdbgOaOn2Ik/Q2RZ3VTgdxaOZR5/fTWyn1kFNe53fBuRwHzv+XZZV3Gk2xtnG65qc0R56KYEEzNfo6pbVCM5uXRr2R89r0Lmup92ock1VRaL2ttY57lFIOs9EM4Khy2sNCKbUEzWb6o4h00iewfESbhLy7lGqXoe1l7qGUikcb1g9EM6VsKSVPIpptukpQSiWjmTgmlnRdRNzQeuEJaL3vC6ljKdrH5UfRtrAFzcT0oIh0Ew0vEbleSnD7rSBVXV6ZXGLvSyjwqGg7Uo5A++D8ppsEV6NtDuauT8DeA3xdUiG6UrEDkzk/r6w5wBMi0kREvNFMP98ppawX/pMArXduRfvIW0RkIsU/rKWiP//9QFf93IbWqXhZRDxFpDXafIGDBUCsiNyhP0cX0ZwTWpWQtyXnOo2cTVnl2dHk4X8iEgogIg1FZEBJBYnIUNF2JTWJSAjwLrBFH9U46AX8Xs4zWUo9kPPaHMlkok0mrRORbLSPxQ60yUBnvkTrVc3iXIYDvwHfofWIdgCd0Xqt56CU2gdkoX0sHHb+Q8Aq/cUric+B1lL+XMj58B5wnRT30kgTkSy0j1R34MazTCbnhVJqMdok5S8i0kn/4N4HfIjWCzrA+W+T61x+lZZXAS6l92UdmvfhabT1EsP1zgloijUabbTxI9q8wuIyypqF5hZ/Pos0v0BTSsvR9mrJA8aXmaNiLET7cO5DM23lUbLppjQ+pfj80yNoZp0EtFHtDMcF3aTYH21e5IR+z1toc0COvH56+mw0xZpfWsUVKO9ZNBlYKyIZaO9UaXNNDdHMaZnAdrSOwM2OiyLSBchWmitzmdQHOTf2kzEwqEOIyFjgXqVUjyoq707g/qoqrzbRR/lb0LzpKuzEUcGy3wLClVJjyr25mhGRH4DPlVK/1XZbqoI6t+jOwMCgahART7Q5wKm13ZaqQHdmqPT6MdDMr4Ar2kiiC5rZ8d6qKLuyKKWGlX9X/aHOTPwbGBhUHfp8QBKa+fWbWm5OXcQHbV4mG80LbzKa27dBFWOYywwMDAwMqg1jJGNgYGBgUG1Uy5xMgLu7auBdURfxSlJ4ZgHzaZ/omqmzmgjOPKIduLjWajtqml3JyaeVUuUuXKuveHgFKt+A81meY1AbBPiZAbCYQR09qCXWsCxejLJQLUqmgbc3c28aXP6NlcSaoHk/fn39T1jTL2hJSZ1jzMp7ALCEl7Q27OKk7RdfHi3/rvqLb0Aktz/8S203w6AULH5BDL5a80iPNR8iccIEJCoQcy3I4MUoC/XSu8yhXCJ7tuU1+xNwkSgYgC97fM6YlfdgS4irlZfcwOBSYshgf8CGq0Vo/stE4pfvQMxmzCGX3HZN1Ua9UjIO5QLax5jzCU9Yjwh6cxLJz02o7WYYGFzUaAoGQnyFgnFDiefSsiDUFPVGyTgUzJc9ztn646Lj3QWBjAFjNGNgUA207BhJywZZiAjBPlAwbihgKJjqol54l11KCsZB0JuTqi4yp4GBAaCNXlo2yCLE11AwNUWdVjLWhLhLUsGANpoxMDCoOpzNYwHZBykYNxTBUDDVTZ01l12qyuVsDJOZgUHlcJjHXC2CnyeEfmtM8NckdXIkYygYjaA3J9V2EwwM6jXO5jE/TwjIPkj88h1YwqMMBVND1LmRjKFgDAwMKotj7Ytjch8gavc8Dk75qkI7pBlUHXVKyRgKxsDAoLI41r6E+J5RJwXjhnIQY/6lNqgzSsZQMAYGBpXBeeX+2QoGDAVTW9QJJWNNiKPpI6N5aWvv2m6KgYFBPeTsyX0HhoKpfWpdyThGMIaCOZfk5yYgZnNtN8PAoE6jmceyio1ewFAwdYVaVTLWhLgz8ccMSsTwgDEwKB3ntS/OGAqm7lBrLsyOEYyhYErmycEptd0EA4M6jaFg6ge1MpIxJvnLJ/m5CYarpYFBCTiUi7N7sgPHKn5jAXPdBrX9XAAAIABJREFUocaVzMWkYJRSJCfuoyA/i+Dwlri6eVVp+YagGNQnsjISSUs+in9QY7x9w6qljtJGL3BmBGPITd2iRpWM8yZj9X0PGGW38/v3T3N0/yrc3IOwFqYx/J7ZBIY2q3TZTw5OIXllFTTSwKCG2LV5Hn/PfxVPn0bkZB6j9w0Tad1xaJWVX5p7sgNjBFN3KVfJiIgbMAyIdr5fKfXq+VRkSzoBwLpHF2LdHH9ejayL7PnnZxLi9tG+12zMZjdOHvmJhT88x6iHvq902YaprG5SVbJwsZGTdZq/579Cmyum4OnTmJzMo/w9fzzRsT3x9A6udPmluSc7MEYwdZuKTPz/DNwEWIFsp7/zQtlspL/5A3suAgUDkHr6ML5BnTGb3QAIDLuKtOTDlS7XMeFvCEydpEpk4WIjI+047l4RePo0BsDTpzHuXhFkpB2vdNlnxx47GyOSct2nIuaySKXUwMpU4jCT/bQgrTLF1DqFBTkU5Gfj6R1MUFgsu7ZOoUHTW7C4eHH6xBKCQmMrXYcxiqnTVFoWLhZstkJys1Pw8ArELyCKvOwEstL34e0XS1baXvKyE/ALbFSpOsqafwFjBFNfqIiSWS0i7ZRS2y+kAttFMtG/evF7bFzxKSazG/6BjRhy5zSaxF7BlqW34eLmh9lsYvg9sypVh2MuxhCaOkulZOFi4diBVSyY8ygoQGDwqA/oP+xNFv3wNK7uQRTkJdN/2Ft4eAZccB0VVTDGCKbuU6qSEZHtaK+RBbhLRA4B+YAASil1WUUqUED6mz9APRnFWAvzMVtcETnzch/cvYQdm36iY+9vcHEN4Nje6Sya9zxD7/qCrtc8SH5eJgFB0ZgtrpWqO/m5CYbQ1EGqShbqG3abFQCT+cxnIi83gwXfjCem/UT8gzuQdnoLC74Zz90TlnLPMyvITDuOj39D3D18L6jOstyTHRgKpn5R1khmcGULL4pJVg8UTGb6SX6Z/RBJCTtxcfHg2ptepVX7IQAkxm8jMOwaXN203SrDG9/EjjXjAPDxi8DHL6LS9Y9ZeU+lyzCoNiotC/UJu83Knz+/xK7NmhNL644j6HPTK5jMFtKSD+PmGYp/cAcA/IM74OYRQlryEcIjL7tg5QLlj17AUDD1kVIn/pVSR5VSR4H/Oo6d08oruL7FJJv/9SO4e3eg+6BFtO72Pn/P/y9JJ3cB4BvQkKy07Si71rNLT96Kj1/VhXt5wfQ/wBCcukplZaG+sWH5Z8Qf3U2Xfj/Spd+PxB/dzcYV0wDw9g0nN/sk+bmJAOTnJpKbk4C3b3il6qyIggnIPggYclLfqMicTBvnExExA50qUvi6RxdCPfAms9usnDr+D90HvY2ICS/fpgSEXsHJY1sJiWhN6w7D2Lf9D7atfgA3j1Cy0/cx9K6ZRflzspL5fe5THD+yDg/PIPoMeZWmLa+tUN0WvyDif91hCE794IJloT4Rf3g94Y2HYXHxBiC88TDiDi2h6zUP4e0bxpV9n2DtX+PwCWhJZuoeruz7BN6+oYC2fmzlonfYtu5rAC7rdjs9+j+NmEruz5bnnuxM4gQjYGx9pKw5meeBfwEeIpLhSAYKgM/KLLWwAKDeuCubzBbcPPzISt+PT0Ar7HYrOZkH8fK5sej6kDHTOXFkA/n5mUREdSjm/79gzqMgDenc5weyM/bz+9ynGPng9wSFxpRb9+2/DjG8yeo4lZKFeoiXTyhZGfsICu8BQHb6Xnx8Qoqud7xqLI1jriL19CECgpsSFNa86NqWNV+yb8cyLusxHYB9W17CyyeEjlfddU49pUVPLomixZZGwNh6R6lKRin1BvCGiLyhlHr+fAuub95kfW9+ncU/PI9/aFdyMg8RHBZNcEQrpk/qTU5GIiaLK1f1e5wOV44tls9ut3Hi6HquGPgHJpMFv6D2BIZdxfEj68tVMo55GMObrG5TWVmob1zV73HmfDKC3KxDAORmHaLfkLn8+OU9xB1cB0BUs27cdMc0TGeNUA7vXUGDpqNw89BGNg2a3sbhvX+do2QqYh5zYKzmr9+UNZLpqB9+73RchFJqc2l5k/3L78HXNZq3GUBQSAwn4zbj6X0r0c178dlbPfDybUtshzfJztjP8t9fIygslkbNrizKJ2LCxdWL3KxjePk2RSk7ednH8PC8vsz6HO7Khpms7lMZWaiP+Pg34M7HfuPI3qUARLe4hr/nv0pC3B7aXfk+AHs2TmTh/01g0C2Ti+X19PInN+socDUAuVlH8fTyL3bP+SoYMBRMfaasORnH2+MOdAb+QTMRXAasA3qUllHpro/1gT9/epFdW+ajUHh4+jP60fl4ePhhtRaQk5nI5T1mYjK74e4ZTmDYleza/EORkok7uIaVCyfj6ubLjrWPEdqwH7lZh/Hy9qZpq76l1vnk4JQ6tSHZ+pMnmbJuE1kFhVwT3YhxnTuwJTHxnDSXUuzqlwAXLAv1icP7lvHbnMex2ayYzRauG/UeTWJ7AXDs4Foat3oAL1+tA9m41QPE7TtjKczKSOSvX14h6eQecrOTyc44gNniTkbyJkY+OBcoP/7Y2UTtnsdBarYjdionhzdWruVQahpNA/x5vscVAOekhXqWM4FkUERZ3mW9lVK9gaNAR6VUZ6VUJ6ADcKCmGlidbFw+jR2bfqBZuydp020SIr7Mem8QACaTBTGZyc9NArSIy3k5CbjpLppJJ3fzy9fj8Au9gabtXsDbtxnCCbpdcwfD7vkSs9mlxDpbdowsWtVfF+zLe1NSeGLx3wxIc2Vcrj/r9xxh4rKV56T9b93G2m5qrXEpyEJudiq/zH6IkMjraHfle4REXscvsx8iNzsVALPFlfycxKL783MSMFm0d9xmLeD76aMpKAyiabsXCGt0I7mZe2nZrjOjx8/HPyialh0jGXy1DVeLVEjBAByc8lWNdsQKbTbuXfAHASdzeCwvkICTOdy74I9z0u77dSGFNluNtau+UxHvspbOK5yVUjtEpH01tqnG+GfdbMIb30Rwg2sAaN7hX2xdfg8J8dtYs+QDPDyD+WflA0REDyMrfQ/5OfF0v3YOAAd3LSak4aCivE3bTWDPhidp1eHmMuvs9sGAOmVf/utoHH3tPvQwaSvfHlHBPH40joH4Fkt74eBhnunetTabWhe4aGVh+4Zvsbj40LjlfYgInj7NSDq+hM2rZ5CdkYTF7MqxfTPIzTqGyexK4rFfuW6k5nqffOoA1kIrUbH3FOVNS1pJZJPuePuGndcEv4PamOg/mJaOLd/KHYQjIjRRrqzIO4YZKZa2Li+eg2nptAwKrLG21WcqYv/YLSLTReQaEeklItOA3dXdsJqiMD+16NhakA4K5n0xFrNbe5q0eQo3j1BSTq3GZHLBbHHBZssHwOLihrUwvVhei8W9zLrq4kS/m8VMppzplWVgw2KSc9Lc6ohpr5a5aGXB1c0buy0PZS8EQNkLsVlz2L3lZ1KSc2kY8xDBDXqTcmoNBfkpWFw8MZm0d8JiccNamFMsr7UwB4uL23nNvzgI/XYiUPNy4mo2k6NsWFEAWFHkKjt5Z6VlKxtuFkMeKkpFRjJ3AQ8Bj+nny4GPq61FlUDZ7WxePZNjB9bg5RNC977ji63GTzl1kHVLp5KXm4mLqzvWQitZCcs5uMMTD8+GxB34GqUKCWpwA+GNbwCgRaeX2bXuGVr1nM7+ra9yZN9S2nQaQeuOw9i86ksO75yCm0c4CUd/oMeA0reSdiiY6rYv25Xi6x27WB9/giAvTx7s1J5wr9I3U7spphnfbN/FZ4VJhCkLP5vSeaDj5czatrNY2iOdOldru+sJ9UYWQIsx9s/abwC4vPvtxRxWCgtyWPPnFJIT9+Pq5klGWiJ2ZWPHuqcIiehN0sm/UcqGzWqjSZvHERF8g9qzeekdRDW/g+yMq9i24f9o1ro/ASFNadC4A3s2/YuAkCtJTVpNg8YduGus5iNxPgoGIH551a0bW3viJHN3aP2AW9u2oluD0qNzNPHz5fLwUF5NSKCr3YP1plw6hYUCwquJZ9Lah4cS7XvhkQ0uNcodySil8pRS/1NK3az//U8plVcTjTtflv76Gv+s+xE37x6kZ5iZM3UYuTnaSCU9NZ7vPr2F3LwgXL26E3dwE2LywT+kC9npB0mMX0hQeC9cXL1Q9oKiMu22AhDtMdnthYh+7OkdzG0P/0TDRmF4e6czcMSbtO18S4ntqikFA/D2mvX8smUX3RIVpsMpjP75V9LySv/vCvLwYM6QwYS1bkhGjD+vXtuTO9u2OSdtaGzzUsu4VKhPsnD0wEoWzHkMZWmNsrRmwdfjOXpA2wnPbrcxb8ZdxB0+gLtPL04npZGWcgIRV1xc/Eg49isuLn6YLZ7Y7VZQ+qhW2VC6DCh7QdFIRkS44bYPuazLdXh6nOayLtcx7euvMJlM561gHGayqmDN8RM8s+RvWpwooMWJAp5e/Ddrjp8o9X4R4Z2+vRnQqRUpTf0Y0KkVk/tdy+R+xdPe6du7WGxDg7Ipy4V5rlLqFqfggMWoa0EBld3O9vVf06nPd7i4+hPMNeTnxHFoz1+06TiMPf/8TEB4LyJjbsdut3P6xFLST28mN+sYIZH98fFvxdE9n+IX2IjTJ/7C4uqPu0cER/d+gV9Qew5sm0RG8jYO7wvC1d2HmNb98fYNpceACWW2qyYVjF0p5u7dxwxTE/xMZnrgw3FbAsvi4rmpeelu5SGenjzWpVO5aZcq9U0WALau/oqo2HsJi9IcWUTMbF39FY1jepBy6gBpyfG07zUbETOFhVmkn96MiIm8nJNExozi5JGfcXF1wyQm9m15hcDwXiTFL8Zs8SL11Ebi98+gQXQX1i/7hE5X3Y3Z4lq0FsZhIistwGV5VJWZ7Nsdu7nTHkhfkx8AJruW1r1h6fM8FpOJ0W1an5NeUppBxSjLXOYwCdSr4ICC2enYBEr/JiiFFgUEdq59ioL8JKJbPUBm2h6Sji8iOWE1FosXfiEDEPNWTsXNp2F0V2Ja9yInM5njR9cQGnU9VnsES356hezMJC7vdnuJbXAIWZvfXiKeml8L42wtNiHnfhUNzpd6JwuKM+87aEpGFb0JSh+dCyeP/MKR3VOJih0LQNy+mRza+TF2WzZRsWOx262cODQHN3c7EVGNKCgIJSHuZ3yD2uHi2YXd/ywj/vAGbh4zHRG5oDkYB1U5igHNI9TkVKIZQSlDGmqaslb8n9QP+wArlFL7a6ZJF4aYTLTuNII9m16kQdNRZKfvIzNtJ01aTGLVoskc2LWY9JQ4XNzCyEzdRqc+3+HmHkx44xvIzzlJevIWQqNGkpsdh6dvc2zWLGLb9icl6RBH968gIKwH0a0eBMDbvwUblr5WopJxFrI/dh1i0d5DBCelc2vLFni5uLD0WBzrT5zE08UFu1IU2Gz0bBRF14jKBRgEMIlwc0wzXj98gpvtfhySAvaY83ktMrLSZV/K1DdZALi820h+nzuhyLx7bO+nDLrlHQ7u+ZP1f0/FVpjNwW1vk3p6C01ajyO8sRZCyWLx5OieafgFdcBakIGIiQbRI3B1SSSm7QBWL36X/Nws2l31GiaThdDIAWxdNhr/kCx6XxFVagyygwf2Mn/edwDcOHQkTWNitbQfvsVut+Ph5U3qzn3EREZxc6gdcxWsyRrepiX/PrUck107/9KUzH/b9Kx0uQbnR0Um/qOB0SLSGNgErEATtK3V2bALwS8gipysXzmy+1NQhbh5+LLwh2c4ceQfIpoMw2rbSNy+GSilMFvOSILFxRsv3xhOHVuAf0gXCvKSyEzbz4bl08hMO4WHTxPMFu8z91u8semeNA4cgf5AUzBfTv+IOUu3M/LWW9i5bRtjF//JgAYNmLttF/3tPmxVeexTuQwUP57Ze4AJV3Xj+mZNK/0Mnr/qCr7w3s7C+BMEe3ozu2tPAj3K9nozqDDR1BNZ8PIJw2bNJ/7AVwDYrPkkndzNmj/fJyJ6KIHhTUiM+xWTyRWz5YxjiNnihZjcKMg7TX7uKfxDupJ47Cd8/SNY8PXDBEX0xmROKRoliVhwdfMgKiCNEN+Sd8LcvXMb994xhFtGjADgjlsG8sLLk/jvc4/RN98Npex8ZU+jv/jx/fFtrI8/wVvX9qr0vEfPqEhe7d2T77Zr0dT/064nV0cZHa6aplwlo5SaCCAiHsB9wATgPYpbZWodpRRr//qAy6+ejrtnBEopdq1/iqP7ltOx92zcPSNo2GwU21Y9Qk7mIXaueRIvv+bYrDmkJK6l3ZUfoFQhB/55mw7XzGLbyodJObWPjr1nY7flsX3N43j7NcfdM5xj+6YX7TUD5wb6U0rx4Xtv8Ov8+URFRaGU4o5RI/lk8xY+lMaEm1xQSvFv+3EaiivPqDA+2rC5RCWTmJXN2+s3kFlQyI0xzRgcU7YisphM3N/hcu7vcHmVPl+D+iMLAOuXfkxU7F00aDIMgBOHf2DD8mk0bnEfDZoOB8DdM4Ijez7j0I73STu9CYCUhJWERg6gUct72fTXbYQ3uh439yDiD86mcYv7CI8ewraVD3J410eENLiW1MSl+Pl70bl9q1Lb8sWn7/PIww8zdswYAEJDQ/ng9YnckufJjSZ/EAhUFvaRx6sqgvvjj3IsI5PGfsU9uOx2O+9u3Mye5GRaBgXxZOeO58ROO5teUZH0MhRLrVLumFREXhSR34FFQAzwNFDn/teUsmO35eOibywmIri6BaOUvXiaewh2Wx55OScwWzzIzz2Fh3dDvPya4eoejM2Wi4jg4dOwKK+nTxNadX6NY3u/4MA/r9Hq8j70GPA0Fr8ghgz2R6T4Kma73U5Bfj4hISFF9YaFhlFoVwRwxiMnEAt5KAKxkJqff85vSszK5sbvfyTl6GlCTubwyvJVTNm0pbofpUEp1BdZACjIz8HVLajo3NUtCKXA1f1M9HBX92BMJlfs9kJt7lLZUcpGWOPBmM1uuLj6YLPl6nlM+v0WWnebRE7GQXZveJ5mzRRfz/0JF5eSI1wA5OZkExYaWnQeFhqKNT+fQCfdHCQW8rDjJiYsdsixFp5Tzoh5v7Boxz4aJuSzaMc+Rsz7pZJPyaAmqIi5bChgBX4FlgFr64rbZtLJ3Zw4uglP72Cate5HdIs+7Ns0EU+/llgLMjl9YhkWFw/2bPw3Tdo8Qnb6flJPrUZMFtpdOQVPn8Yopdi++hFOHvmRlMTV+AV1JOn4n2Sc3oinTxj7t75BoxZ3k5+bSGFBKsPumU1kdBd99GI7Z4IzPy+PJQsX0LJ1W5597nkee3Q8O3ftYtmyZXQPD2fK6VOMVIEcVHmsV1n0Fh+m2BLIx8Zba9fTKiiQPJuNbhERvLtxI+1w50VzQwC62L15458dxPj7Y1N2cqxWQjw9SMzOIcjDg2sbRWE2mdibksKWxFPF0gyqhDorCzlZyRza8ycATVv2ocVlg1i58H2yM7SoN6fifsNsFg7teK8oQvKR3R9jcXEjvNFQIptr84seBxpxdPc0vPxisFnzsVlziD8wg9CIWI7s/rgob17OCfpdfyPv/G9Kme1at3o5Pr5+THpnMhER2hqVd//3Hlf2G8jX838hpED7BM20JzFQ/PnWlky6svLuhk1c37QJeTYbTf39QAlxGZnMNDfFU0zcqoIYk3GIyes3cnloCKdzcwn28CCjQFt+cE1UFIEe7qTk5rE0Lq5YmkHNIhXxthARH7QggD2AW4BEpVSpQQHDIi9Ttz9cvb2MvdsW8OfPLxEYdhU5mYfwDwyhebtBLJ3/HwJCu5OdsZ/Cggx8gzqQmrgKFFhcPbl64FP89fNErhj0ByaT1vvau+kV0pK34OIaQH5uIsHhLbj2hn/jExDJ95/dTlbGCUxmN3oOmkCHvg8z+GpbiXuQ5+XlMnbkYLw83AgLC+Ovv/7Ey8ubSE8L40NDaBnTgjdWrWXJkaO42cEdE3kocrHTCU+OUkAGNjpbvNlADqHeXrRPF+40ayOiU6qQ8bajhIgLmdi4XDxZZc+i+9VXkXr6NP75eVwXFcmbq9bSTbw5Qj4hQX58OLBvnVY0bb/4cpNSql6s9jxfWYDql4f01Hi+/Xg43n6am21W+i6uHzWFn2ffj5dvK0CRnryVgJAryEjdht2ah5gtRDfvQU5WKr7B1xEUoUVNTj65ggPbJ+PmHkxeTgKe3oF06nE3l3UbzW/fPsqR/SsRge5X92bqZzPKNFe9P+k//D7//7jyyu4sXrwYi8UFH18/br39bkbdeR/fzPyMLz6YRGZaKsFYyMFGHoomuOEmJrarHLqZfdgv+cSGh7AnPpHpliZF5d9nPUyQWDig8uhm9mG7LYfgxpE0ax7Dlg0beLtHd55espQWdjcA9pry+eqm62ng7V1ak2ud+iQLFaVcJSMibdHidvdCi0AbhzbZObG0PNUtVEopPv5vJ1p2fgtvv1iU3cqOtePJzYqndbfJRWnbVj9CZLPbKSxI58iuj7hh9MckxP/DhmWfEtygL41a3EV2+n72bHqJyJjbyc08QHBYMANHTCqxXofnWGkeNN/MmsbqpX/w2aefICL89ddfvPPue/z4x2oKxg0tcmX+bvde3l2znpdMDfiP/QSPmcLJx44vZmbZkxhhCsKC8LFLMjn5hbxsbkgYLnxiO4WXmHjEFMYTtmMEYaY9niyL8mL+4oUMH3ITh/cf5L80IEbcsSnFs6bjPHh1V/pGN662/4/KUl8E60JkAapfHn6f+zQ5OT7F3JDTk1fgF3R1UdqxvTPJz0skqvmdbF46hu59HsXLJ4RVi94F8aRlp/8AsHvji5jM7oRFDSTx2FzuenIJLq4eRXVV1EX5ePwxRtzQiyWLFhEQEEBqaip9+/fn+/nLaBh5xkEgJeU0/TrFMkaC2a1yCVEWYk3aaGO/PY9UsXG3KYQH7IcpVDBWgrja5MsKewbzVCpTzdH8YE9hjcpisPgz2zWD+YsW8n/ff8/8WbPonu3CbaKZDb9RyWQ38uW13nXXw6y+yML5UBFz2VtopoEPgA1KqXONpTWMstsoyMvAy0ebBBeTBQ/vaDJS9xZL8/RpSnryZpKO/4m3fysWfDMes4sfNmsOhfkpbFl6Jy5uAfj4teDEwa+JaTuIPje9UmKdFRGu5NNJtG7dqsgrplWrVqQkny66bks6gTmkAbe2asGhtDRe2r2PXOx8YE+krXgQpwoARZqy0tXkjdVmp1AUL9mOU4jiavHmAVMoZhGixJVDKo9ZJGNOyMBisRAb24Kde/bSxKz13MwiNMaVlDJW/BucF3VOFgBysk7j6d+h6NzTN4ak43/g6XtmAa6XXwwZqTvYufZpvHxj2LxqDoWF6ZjNXvgGNmX7mkcB8AvqQOqpdWSlLmX43bMuSMEApKYkExHRgICAAAACAgKIiGhAakpyMSUTGBjM253aMnHrHvKshXhh5oDS5ifjKSBSueArZsJc3Ek3KWblJfO57TQRuPCKORJ3MdFM3PlTZTBTnUYKhOSUFFq3acP3BVaaKm8cS2WaKjeW5uRe4FM2uFAq4l1W9u5bNYxSilMndhAYGsuxfV8QFXs32RkHSDu1FrPFk6N7v6BRCy0tJWEVoIi5/FnST2/GZsvjsqumsGPNk3h4R9Gi08tkZxxg78bnGTXuR4LDW5xTX4OmrkjaWo4eCKVzx3YAHNi3h78W/0Zko2gioxqTlZVJm3bt6XLFVfzryQeJbd4cTy8v5s6dS5NmsezauY0TnsLB7YcY6OZDlK8Pz3fvxvPdu9Hpi1n8yxxBW/GkwG7jPvsRNqhs/rHnYTMrYnDnLlMwH9kT8VYmLAj7VB5bVQ6vmSM5pgr4sDCRf/75h8V/LiHay4fpuUl0VJ5kYWedZPNgWOg5vwsgOTeXPSkpBLq5/3975x0eVfE14Hfu3ZZN7wkkBAi9SBFCkV6kihQFUYqCiIKABbGCoIIiiqI/OyAqChZAP1GKICIoIr33TiAhIT2bbHb3zvfHbkJCDSUQ5L7Pk4e7NzNzT3b3MDPnnDkHp9TIynNQIyQYf7O5JD/Cm5bSpgsAaSlH8fWP4PihufgF1QLg5KFvUQ0Gju+bXXDv+P6vAUFgeGP8Amuzf8tk6rScScrJlZxOWEW9ljMB2LNhLE3bjaRBiyFFntO9awB5eXZO7F/PYaBOvYaYzGayMjNY8P3XAMQ1bkZS0iliKsRSIbYyp0+f5vNZsygfE8P+Awc4efIkKSnJ/LVqBdu3bKBeg8bENW5G87AQVg3oS6e5P1A/x8gQ1f19fdd5km3k8JUziUQljzzNxWtKFL/JdA7IXPxQSZcu5mkpdFUCaCn8eMR1iD27dzN71ixCfaz8kJoKnnMy80QaXaLPHwWX53Kx5ZS7rEewl4WTWdmU8/Mj2u8K0xboFFAsn8zlUlLmAU1zsfDrxzl5fDsII3ZbIi5nDopqomz5OI4dWIVfUB0yUrdhMHjjdOaAdGIwBeDlHUWu7QTBES2IqtSPPRvHk5m6A0U1EVWhET0fmnVOXH6FiB28PHog1apW5fDhw7Ru15mYipV4Z/J4qlSpwqFDh7BarcTElOfwkSP8b/pcHrv/brTMbAJRiSePchZfEu3ZSCkJV0wc1/J4vXUL2leIwaVp1Jv1FQtUd16wN7WT7JG5+KNynDxcqkqUS8HuiUA7SC45SLxRGKmEc4fiS7J0MNR1GKNqwAeBGUGCK49oTJzCyW2RYXzUsf05f9v6hASeWPo7McLMQacNi1CJNJg5iYNPOnegSlDgNf/8LsR/0URQmJLSh61r57BqyZtYPbv4/CzIFmsQ/sEVybOrpCWtBySKasHlcmC2BKEavNBcdhTVTI1GUzi+/2sSDi8ABBZrEANG/YrV50xkWveuAWSkpzJmaDdUxX1qXpMw4fX3GDKgByHBwWRmZZGVlcVtdeqyd88eRo0ey/LFC/n3z9+Jxsgx8vBVjXgJlVPOXKJVL467cmnfrRf60raDAAAgAElEQVSvOE9jiIjmkYWLaZus0kjxYbGWxiwtmTIYOUoeUlGwSnBJqIiJfdixoyGBriKAQUooqhAMcR7iFA4sQqWCauGA00agZy3tMCrMu+dugry8iryPabm5DFq4GHIcZLicpGtOqpm8OeTKZVTc7dxb/dzFZ0nxX9SF4pjLSg27Ni3gdNIJ6jT/nI0r+lG1/ssEhDbE5cplx5ph+AdVIDC8CTUbT8VuO8GOf57A4cimSt3nCQxrhNNpY+uqRwkKb4KPf1UMRn+q1H+RHWuGcXT/KmIqu221+RX87uv4GG9NeZNWLVuSlZVFz3vu4cd53/DBBx+QnJzMt99+y6SJE8m129m6dSvDB/UhKDuPN9UKGIVgqSuNpbkZTFcr8LTrKAMIJllxMn7VX9QICcIlJRV9/fgxOxU/qZIinXxWqO9nrmRUVD5UYwrufS6TMaFQVVjQpGSelopJKJQTJl6TZRjqOsTzShkaKN7YpMbTSceZs2sPPatUwmI483E///ufPKGFkYYTh3QyUYnGqAmWaumMX7mab3rcdaM+Zp1ikJ2ZxJ+L36B20484eegHLN4xVKjpzn5zeOc0NOcxwEDDO38EYN+ml8lM24t/SD1iaz8NwIFtUzm+70sCw5qQdHwZ9Vp9yfF9M/h72TTadX8FOGMim/PZ69SvV5cJ418GYNzL4xnxyP106tiJUaNG0rFTJ2Z/9VWBfAMGPkheVhYfqDFEChMnZR5Puo7ytlKWOSIFVROMUcMY/n8/0LxeDRr4BVM2wI9fUk5STjMxS0vmHbVcQd/hriNkIflQLV/knkRSX3ijCsEGLZvTOLEIhXeVaH5ypVFO+PGY4t4ZfaQl8dpf//DSHU2KRJn9b/0mKmcr9JHhDNOO8J5ajkjNxEnyeOrf9bSOKUeItejEpFN8bqpJJj3lKL6B9RBCwZ6TREBoA4RQMBisWP2qUalqTXZvWcjx/V8BGi07v8jyn8YSEOpeGBgMVrz9K7Pz3+fxC6pJtdtfxWCw4htQi7SUY8RAQWhykLdGYkI8zZu5A4d8fHyoV7cuBw8coHmzZkx77z2cTif9+vcnODiYrKws0lJTaCt9MSruXUN9xZsvXaexCoXqwkICDuKEN5+6kui54GcUl4avojJHyyQPSW8RhFGc6etyJdFQeBe594XrNBWEicGuQwjAjELb2PJ4H0xDUSAZJ/WEOyrBKhRiXUY++XcjX2zZxqedOxDj74dL00jMzaGeamWOdpo6wnrmGcLK7Kz46/aZ6lwZmekn8PKOxMu7LLm2k4SXu6sg0ss/JI7M08kEBIex7jf3oeHKtTqTk32UwLAmBbvawLDG7Nv8Oknxy6nW4DVMZj/8Q+JIO70UKOqDOX7sEA/0uaegb8sWzVm8eBGtW7ciISEBPz8/Bg0eTNmyZYmPj8fLyws/Wx6RwgRApDARjpFk4aIB3iyS6UQKE8HCyKt7DpO3eRdlVTPxzlyGkkUYxiJ9gzGQh1bkXhmM1BNWJmonCkxibSvEsC/+FJGaiQQcdBL+BTLfrln54NgJun43j7HNmtDJc/j5WHoGbaQXycJF+FnPDVdMnMzO1ieZq+CC8YdCiJ+FEP93oZ/rKWQ+YWVqknpqFU5HNlbf8iQe/RUAe84pUhLXYLH6M+CJRQx57i+Gj9vKbY0ewGjyLdIuLWk9RnMgoWXvxGgOcN9LXkdYZI0iSqWqKpWrVOO7778H4OTJk/z1199YrVbmzJ1LQkICWZmZzJ83j08+/pj7+/bFaDbxB1lkShdSSpZo6cQKM8nSwUZpowImlohMoqOiiDB4MU2U4xXKci+BWCxe/CEzivQ1CoWVouh4lYSZx5VwAhUDr7VoxtpB/WkTU441ag42NMph4jfNXUwtWTrYLnN4XkTQMdfKs7+vJMOeh6ooxPr68ZtMp6Iw85c884ylZFAtSK/4V5jSqAsBQTHk2hLJSNnuTol0bBGa5kDTHCQeXYjTYaNzn6kMG7uRYWM30rnPVPwDo0g8urBQu59RDVZ8A2vjG1gdTXNw+sRSIqJqnuPkr1b9NuYvWEBeXh55eXnMX7AAKWHO3LlkZmaSmJjIjOnTeW/aNGZMn05aWirJWh67pNvRvkvmcAoHIdLAb1o6sZjZJXPIMqm4nE4mUoZnZTivUQaDopKCs0jf0zjJwHXOvR4iiNsVbzrHVuSfgQ/wUtPGJGkOdskcKmLmNy0dh5Q4pOQ3LZ32+DFRlmH8qr/ZfToFKSXVQ0P4XckiRBo4haPIM5I0B+V0v8xVcUGfjBCi5cU6SilXXuh3JWWDllLy56LJbF07G01Kd3lWgw9ORyY+frE0aHHfOUkrD+xcxi9zRyEUIy6HDYSC0eSN5rJjMLr73j1wPCNHDjknNHn/vt0MH9wHl9NJWnoaw0c9x5RJY7EIBZMU5AqJajTg7eeHLTUN1aWRB2hIzAikouBlMpFuzwV55p5BCHJdTgTghUIOGiruU34aYEbgRGL298OebcPldGJGwYmGAUEukubRUbzfvk3B+/LOv+uZu2sPJqGQp7kwC4VszcUAJYT6wsrLrnjsSJwKDKtfl+bRUQxb/Bt5dgeZmhOX57kIwSddOlAnLPSaf34XorTboa9GF6Dk9OHQnhUs+vYpXBpIVx4oKgKBxTsK/8Ageg/5qkj7XFsaX0zrRK7NvQgRQqAafXA5s1GEghAKkTG389ns2ZgtXkWiyHJzc3h6+ENs2rgWgHr1G5GZmcbuTRtxOtzfZcWg4h8WSvqpJDSnC6koaJoLCwp2AaqqICS4XE6MCPKEQFEELpcLAfijkobLnUHaLSEWBDlIMBpQVRUt1+45X+b2x6gI/L0szOtxFwEWtwnsz2PHeX7Fn5ilIEtzIhBoUlJbePGECOcdmch2mYNRUagbHsak1s15YcUqNp86Ra7mAnlGLwfXrc3w+vW4XpR2XbgSbirHfz45tlRmv3cXkRUH4hdYC4lk97oxdLnvLaIqNj6nfV6ejcTjW/EPisZospJnz8bqE0J25inu6RGDj68/oX5uh2Z6ehre3j4FaTIcDgcJJ+MJCAjE18+fuEphPCgD6CgCSJYOnjGewuznw91JGs0UP7Kli2eNSeDrxfPPPkvlypWxWq30HzCA9FNJPEQQARiYrZ1mshqNDwrfaKfZSy4jlHAedx3GaTEzYfx44uLiAOjbuzfSbueRJ56gTevW2O12HurfnymNG1K3ULqOtNxcsh1Ogr0sfL97Dz9s2MEblOUZ11G6igA6qm6Zx4h4pnZsS82QEObt2cvsdVt4QYtAAZaTwZFQE5926VBin9/Z/BcVqzAlqQ9ORy7LfxpHUkIK5ao/AsCxPTMoWy6aVl1fOqe9pmkkndgOQGBoJWxZSXj7hmHPcU88D9xftSBNks2WDYDV6k6gKaXkVKI7IXVYeCT3d2uDeed+XsCdQXyyIYm8KuUw7T3KKIc7cOAdYyqnywRTrWZVRo4YAcDUqVNZ989aKmQ6GUUYw7TDjFPLUl14sUvm8Iorng+V8ryrJbDb6KRpyxaMG+c+ijTp1VfZu2ED0dWrM2HiRAAmTphA+fRURtY9k68v1+nklM1GqJcXRzMyGfjzr0yiLKtlFkelnedUd02ZKSKRGtXKMbJhffanpjHw5195ToYTKowkSQeTlVP82qfXdYu4/C/qwiV9MkKIysDrQA2gwFsmpbz6lMFXiJc1kG79P2LBrMEYTYHk2hKp26T/eScYAJPJSnSh33lZ3ZFTDw5w15py25wPM2LIAxw/fgQpJc+PfZ1e9w3AaDQSXa48AC6Xi2yng/aquwhSiDByu/Bm8alEfsHETNdpQBKTZ6FOh26MnzABh8OBpmk0a9aMlYmJdFACmKOd5g7hg68nk20HxZ9FrnRChJE44YOjYV2mvPUWVi8vMlJT6V2lEtM3baF/v36oqrtP8+bN2ZN4ssgkE2CxEOD5hPrVrMH+06kMOXSYdJy0V87IXF9Y2ZOSSt2wMJJsOTRxWSmjuu3QHaQ/T6XoPpnzURp1wWC00Pquccyb+RC71j4FgH9QGZq2f/u87RVFITzqTI01k9l9SNdo8iowkQV4OXn+qREs/sUdNNCxS3demfw+RqOR8IgzBb+8VCPtpS8Gjw+yncPKpwcOEWl3MoCDANSwW/D39mH9ho107uKOAA8NDaVSdAx37kwiSTgJw0h14fZ5VBdehGEkSTjprARw3JhJWno6PXv1wqQoRJqMRAT407d/f6I8JSx69enD11PeLPJ3WgwGynlKJFcNDuLl5k15afUaDBKGK2EYPH6aNpoPvyclI4Qg1+UiTDVxm+aeVCOFiVDFyNGMTGqH6mH9V0pxco18jruOuRNoDXwJfHXRHteB8LK1GTR6BV36TqH/qF+4486ni923e9eAc2zOTw9/iO53d2XLpk38348/8v7UiezYVjSDu6IoeKkGNkkbADapsVPk4oWglfDjOzWWd9UYjkk7i36ez/Bhw9mxfTuLfv2V7du2EezrxyZpI0IY2SJtOKTbW7lByyYSIzapsU3mUL58eYYPG4bFbuf7zh0YWbcOEf7+rFrtLp+blZXF+vXrifa9sK1YCMGEls34puddhJjMRWWWuQV9o/182araC2TZiI2oUpx24wZTKnXBZPahz9C59Bo0nV6DptP7kTmYzJf3GRbWh+kfvkNK0gk2rF/HhvXrSEk6wfQP3zmnT44jj3+xIaVESskGg51cl9v0+o0ayzdqLC5g585tVK5Uic2bNrF50yZiY2Oxay42GOyESJVEHJyU7pxjJ2UeiTgIkSr/yiyEycikiRPJycri1Xp1mNG2NTHe3qxcsaLguStXrCDKep4UHIXoFFuRhb170SC6DBtEIZmFjXIB7sko0tubRFdeUVlceUR6e19saJ1LUJzoMi8p5XIhhJBSHgHGCyFWAS+XsGyXxGT2JiLq8irfnu/UssvlYsf2Lcz7fi5CCCpUqEDLli3ZvnUTNWvXBeDbr2cy9Y3x2FwOplnTiVHtHLKlo7ncmZR7KIEIISiLiQbCm5UZaQwePAghBGv++puM1DQcLieTyaCM6kUCeQxxHSYYA4exE46Bwa6DGBD8sWwZS7Kz+bhNKyI9/+FPatqIp0aNokrlShw+eow2ZSJpUibykn9vOT8/prRrxRNLf6e8YiFes9O2YvmCvnfFVmTOtp0MSjtEACqJwsUnTe68rPf0FqLU6oKiqIREVLuivmfrxJbN/9Lvgfvx8pwneeD+vsz5fn5B+x3bNjNm1MMknUogNcSHp2zJOBwOEh05GBF0V8KweIql3a0EMk1LLBhv2/bt7N+xk8TUFHYj2KBkYUQw0nWEWCzsJxcvFJ7XjpOGC4vdi969ejH69vrEeb6zw2+rxeBlK7i7cyekBC0zkxntWl/y7wzysvBy86Y89PMinsqJRwKKl5HPG9QHIMTqRd8a1Rm5bQdlMRKPg/631dQjy66S4kwyucJdXm+fEOJxIB44/xHyUs6F0mKoqkpIaBgbN26kYcOG2O12tm3fTptO7loc//y9ks8+eJv5837ggX79ePWVV7BYLPj6+vLkU09hP3KUXeRQCysOqbFP5mI0mti4cSMOp5P/TX6TaUQRphr4SE3hULg/UaeSGeIIIBfJTs3GYtJ5VITyvZpOt8gI7qteDT+zqUDGBhERLLirM3tSUgmOiabqZUSANYiI4Kd7e7j7elmK9F2fkMipzCweVcIwI1hJFl9v20G9iJvyIy5p/jO6kM/5dCIiMor1GzbQrl07ADZs3Eh4hDsTeHZ2FsMG9+GlF19gzd9/Y/HyosOd7kXJD/PmsWjBj2yTOTTCvTjaLm04BazfsIEmTZrw8ICBDMqycodamb9kFp+YM7A7NMa5InAKQJO8xgk6C3/S0cgItPLCHY2JKrRr9zeb+aZje7YmuU/o3xYaikktXkkff7OZuT3uOm9fm8PB/D176S+CKSOMnJAO5u/ey8N1amO9SCkDnYtTnEnmCcAKjAReBdoAA0tSqJKge9cAlv36AzOmvEiWLZtWbe5k3JT/8ceyRbw75RVybDaGDB1KXMM4Dh0+RI1a9WjR2q086/75ix49ulOxYkVef/11nn76aWrVrMmuXbtxZGXjwp1frLawckTacZiN9Lq3L48NH05QQCAtc81EeXwe97v8GZ2agtnPm3FJ8bg8kTSVVC++FenUKhvOkDq1C2L7fz1wkHfXrifL6aRlVBRjmze5oi98sJcXTcueuyJbl5BAW+nLHYpbiWOkmdEJJ67wXf7P85/QBYCUUwdY/eNTfDphH7HlY5n0gTutzEtjhrN/324Mqsq6deuxWq3EnzjJl98tAuDQgX0EBQXStUsXGsXFcV/fvmzetImc3FxOHjyMQ9NYTBq7XDkIBEfII6ZaNZb+tpy1a9dizrbTQgkHoIXwZZ7BTrnGd/DaimU4pcSAIFA1s8vgIEk4+Lx6TSI8E8zBtHReWvEnBzMyqOjnx2utW7jLAFwmJlWlQcS55c4PpqcTgEo39Uy2i+Uym4Pp6dQKCTmnvU7xuKRPRkq5TkqZBWQAI6WUPaWU/5S8aNeO7l0D2L55LR+Mf4In08x8kBdO6vI/Gf1IPya/+gLvvP0Wvy1dwh1N78CW62Dca+8y+d3PCv6jt9my2bxpE1JKWrVsyZgxY1i/YQP2tDTGukKZpVakltGH9JgwHn5uNGUrV6Jps9Z8+9Pv3HZ7Yw6Z3PZfgEPSjsFoJD0piQlqWaarFaij+HDaBOPbtyxSdnZT4ineWP0PTzpC+EBGk3b8NJNWX9u3PtjixSHFUUS+IIvu5Dwf/wVdAHDk5bD4y750OJbIZ84yNNufxMO9u/Bw/x70uLsrf/z+O08+8STx8fH07DuIHxauJDTMPTFYrd4cO3qU1NRUQkND+WLWLHbs3MnhPfvop/nzhVqRB0QwJ8yCOwbdz30PDqB6rTr8sHAlPe8bRLoiyZAuADKki1N2G2tWLmegEsIstSIDlRDSNQf3TJrM/PX7CDGbcSYcIyv+MEN/XUKzdAOfEkOzdANDf11CjtN5zd6XIIuFJFdeEfmStTyCLXoNmquhONFlDXA7PH09r9OBQVLKDSUs2zUh3xywY/3vtHNYCqJYHnL4MWrdGvoPHkT9+m6b7IsvPE+ve3sT16R5kTHMZjNHjhylnyeiZdmyZUhN0kn6U11xjzdcC2FkfDy/LFpESFgkrdt3RlVVxr76FgO3buGl4ycJkyrrZDbh4eVpma4VyDKUEIblHiEusujq6u/4eNpL34JnPCSDGRN/bSO/elSpxP/t2cdLWScJkwbWKdlMbXZp+/atyM2uC/mk5yThleugs3Bb+jrjzy85CeDnTb8H3OfM+vV7gG/mzqV8hVi8fc6YqnJybPj4+NCzVy+aNm3K33//jY+PD9asXDpLt671UINY5Mxi167dHDx0iK++X4y3jy+973+Qk4cPMXr259SRFraIXBo2asnOlSvoorj7dhEBLNDSMKgG93M/dPuCdj7QDi+npLOnXWcRwGIti0Np6dQIOZNn7Woo4+NDn+rVGL17H3WwskXY6FOtWoFfVOfKKI65bCYwTEq5CkAI0Qy3ol2ex/06k59/LL+4mC3HxhHNXlCN/YR0ICQcPnykoM+hw4fx8zt3+30i/jiVKleie/fuZGZmEhcXx7hx40g0444z8oznbfXh4cdGF0wwABaLF1/+tIzlS38hMzODJ5u2YOobL3Ns39Ez40sHRnHuptLPbGaH4izSztdoOqfd1WAxGJh1V2eWHz1KVl4eT0RGnlNbXaeAm1IXzqZdYzPz3szEpgRjFSo26SIpNxtTmkpmZia+vr5kZmaSnJSEr19Akb6KomCz2Zg4cSIn4uNp27Yto0aNQgqJTXMVjJelSBo3b8+U//UnMOjMJDDqhQk0ad2OA/v30KdSVRCCYSt+wybO9M3QnIQVCpUGCH59OmntGmFznWmX6szD13Rt9WFk3O00ji7LwbQ07g0IOGfhp3P5FGeSycxXKgAp5WohRGYJynTZxB9Zz19L3sFuz6RS9bY0avM4XZu7ipzgNxlNHPRRed15mjAnLBUZSIORNWvW8OhjwyhXLpoFC37ktSkfnDP+yfhjHDtyALPZTFRUFPPnz0fTNLYoOUyQJ4hSLKww2Zg0dSat2nY8p7/ZYqFzt14Fr18YP5m7ltdlvCueMhhZKjPoHxt9Tr8elSvxw849vJ6TQJg0sEJk8tpZu6xrgdmg0rlihUs31Cn1umDLSmbFwtc4nbif4PBKtO76ElafM/6E7l0D2LvrKFZfH553JlMvz8g6NQfVYMRsNnNv7960bt2aP//8kw5dehScEcvnwP49GI1Gpk6dSovmzZkzdy5msxmrt4knko/RVPVjo9lJl7vu4+HHnjivjHFNmhexFpQtX5EnjhylEd78QzZlK1QkrnHRYqPR5cpzZ/c+PL/w/6iXZ2CTOY87g8KItKXhtKUBIFQVNbTo5HQlxEVG6JPLNaQ4k8y/QohPgDmABPoAfwgh6gNIKTeWoHyXJDlxLz99MYRy1YYRZI1k99bPKBelAROKpIjxDwykUcvmxDVsyNxvv6W6tQLPPfssGzdtYtq0aZSv/AiffjmfGrXqnPMMRVW4vX59mjRpQmZmJs+OGcNrEycyYsw4NJckJ8fG9BZtztv3fGRmZmD08SGmTWtMZjPlN23EqZ1b/8rHZOKb7l35ef8BsvIcfBZVhurB18Y0oHNFlGpdcLkcfD+9P16+dShb6XFOJ/zB99P702/E/6GqxgLTcdlQb4TZxIgJL7Bs2TIy16zh3TfdhxmfGfMsx04k89gTL9Cuw7mZuH18/XE6nQx95BGOHjvGo0OHMnbcOOLad6VO3fqkpJymeaUq5+17PhwOB8JiolqrZnhFhFM9IZHjJxNwOBwFWTfyGTt5GsvatOPA/r0Fz8j3X4bNHcfxP7fjTDgGFNQpQ404d/Gmc30pziRT1/Pv2WcBmuJWtDbXVKLLZP/2xYSU7UBYlDsSrGKtMSxd+AzjX3mlSLse9zzA3K+mY7FY2Lt3Lxs3bMDb25t69eqxfft2ypWveMFJomq1Wsz/7iu8rFbKly/P62+8gaZJ7h/wyAXlys3NZcRDvdm3bQtevr5MeOdj6jdozIyP32X+d7Pp2bMHL77wAuA20w3ocy+jEo4VlGjOx9to5L7qV3b+QeeaU6p14XTiPuy5OVRr+ChCCHwCqrFl1YOknNrHkCFNAXeocohvLI2atmT2119js9kYN24crVq1AmDcuLH8/OtS2nfsdt5nhIaGAYIZM2bQqXNnZsyYAcADAx6+6CLr/akT+fFLdwRb9wGDGPHUi/yxfDEzPn6XrIw0Pv35J4Rwp3Zq36EjB/btplqN2kXGEELQvmM32p9n/FP3vYLpPvd1YPYBABKfeaZg0jmbs/VMp+QoTmXMUu0FVo0mXM7sgtdOZxam89hpAwKDmPvj78ydPQNFUcjKysLbc5I3PSMT40V8HeERZWjZsiWJiYkcPHiQXr16sWz57xeVq2+XliiHjzFMBrI/J5dH+95Nm67dSUk6QaO4BmRmZBS0zczIwODj9gXll2jWKX2Uel0wmNBcuUjpQggDUjrRXLmYAtwO/vyzMEIIXp/6CT/+8DVfzvyIzMwzFr/MjIyL6oLFywur1UpspUosXryY2EqVOJWUjPkiEVgfvPsGX70/lceEO+nqR+9P5eiRw6xbs5JBgwYxc+ZMnE4nRqMRh8NBbk4OxqvwtaR6xwJg+nD+eX+fv+spDpcuNH31XPvskaWL4kSXhQOTgDJSyk5CiBpAEynljBKXrhhUr9uTjatncWT3x5gskSQf/44nnxl93raBQcE8NnIMmqbx0KDB9O/fj507d3L06DHatOt0wWd069mH2bM+pttdXYmOiuLT6dN5+NGnLtje6XSy/8AevlZj8VFUGuDNPjWPJb/+yLp//8Vut9OjRw8mTZpETExMwXhR6qFif/l1rj+lXReCQmMJL1uLvRvHEhB6B2lJfxFethYD7ok97wHkXn0GEFupGsOH3Fcw0Xw2fToffDb3gs+oGFuFGrXrkpaWRp/evflt2TJq1K5LxdgqF+zz01czeUSE0lJxB5S4NPh8yS+8+OoEunfvzvp16xjyyCN07NCB35Yto1rN2y463tVSeNdzMQKzD+BzdEuJyVGYY9V7ui/KB1y84U1Iccxls3BH0Lzoeb0X+BYoFYrl4xfGF/OX8e0XH2HPSeLOkW/S9s6Ll2IfNupZoqNj+HftaoJDwvl63lJ8fC8cURUaFsE383/jq5kfs3Xnfp4d+8YlnwHgKnItAXda87CwML7//nsGPvggoeFlC8Y7BfBnT303U3qZRSnWBSEE3fp9yMa/Pic5cR/V67Rk/KQnC4qZnY+6t8fxyax5zP/OnYLtk1nzqHXbhVPbCyF458Mv+WrmR6zfspPbG7em/6DHzinvfTZaofW6y3Pt0jSEELz//vs8PXo0n834nF59BhRrvOtBqncsqdVjb7QYNz3FmWRCpJTfCSGeB5BSOoUQrkt1ul4Y/IMJiwjglYmTLquf0WTGaDRhNJoQF1HCfCLLRDHmpdeKJ5PBQPXqtRm7Zx+9ZSD7RR7bpY277+nLkEeGMnjQQ+zYuZNcu4N3PvqSgIAzJ4yjWtTSdzOll1KtC+A2mTVsORSAavWjMBqzCLlEzS1VVQuc7Gox0rOYTCYGPzqq2DL1HjKcj96YULDomq4l0bl7P958cwouz2HKtWv/ZfK06TRt1qrY4+rcHBQnC3O2ECIYj+lQCNEYSC9RqS6D/FDly+HTD97mw2mTqBobzan4A/S7pwPZ2VnXVK5vfl5BxbZt+CrAyZZygXz54zJeeeM9OnXrzY8LF5OSkcvX85YUmWDAvZUHt29Gp9RRqnXhbKqVyTrHTHY2Wzdv4OH+3QkJsBISYOXh/t3Zuvnani19+NFRPHDWsrcAAAe6SURBVP7iBH4KUfkpRGXEi6/w6hvvMeX9Gaz+ZwOr/9nAW+/P1CeY/yiXLFrmCc98H6gFbAdCgXuklFsv1Keki5blc6GElxdDSknDmmVZsngxZcq4TVIPDRpM11796Hr3vSUi5+WS75i8VSJgbpZCTVeiC3D99KEw3bsGFBxEvhhPP/4QTRrW5QHPSf/Zs2ezdsNW3np/5nWQUudsapQPuCl04XIoTnTZRk/52aq4gy32SCnPPdRxnalWPwq49ErtbDRNw+F0EhBwxsEWEBCAPTfnGkt45Zy67xXdN1MKKa26cDb5i69LTTAAefbcoroQGFiqdEHn5ueC5jIhREMhRAS4bc/A7cBE4G0hRPHzzJcQ1cpkXbaZDNw25zs7dWP0M8+wfft2vv3uO1b/tZqmzW/oEYdziB3RD+kqVeb+W5bSrguFMfi7D+sWd/HVsWtPprz9NmvWrGHNmjW89fbbdOzasyRF1LnFuJhP5hMgD0AI0QJ4A3clwHTg05IX7cK4dzEUOdF/Obw6+X1CI2N45tnn+fmXJXz6xXwiy0RdQwmvnvyQRt03UyootbpwNpfro+xy970MfXwMk6dMZfKUqQx9fAxdSonZWOe/wcXMZaqUMsVz3Qf4VEo5D5gnhNh8kX4lTnEcmhfDYvHihfFvXrrhDSZ2RD8OvD/7RouhU4p1oTD5ZrLLXXz17N2Pnr37lYBEOjoX38moQoj8SagtUPiIe3FCn0uE/F3MrYC+myk1lEpdOB9Xs/jS0SkJLjbJzAFWCiF+AnKA/PTmlbiBYZtX6ou5WYlqUUv3zdx4SqUuFCZ/F6OjU9q44CpMSjlRCLEciASWyjOxzgow4noIdzb5EWVX6ou5GdEjzW48pVEXCuN29rv0XYxOqeSiW/3zlZaVUu4tOXEuzq22i8lHzwJw4yltulCYrs31CUan9FKcE/+lgquNKLuZ0bMA6FwI3UymU9q5aSYZ4JbcxeSj+2Z0LoS+i9Epzdw0k0y1Mtc2t9jNRv5uRkcnH30Xo3MzUCKTTEjm4ZIY9pY0lZ2NbjLTKYy+i9Ep7ZTYTmbg6sHX7ExLtfpRt7SpLJ+oFrVutAg6pQR9F6Nzs1Ayk4zRhFBVGr3XgQl1V5TII25VdL+MTj76LkbnZqDEdjJqaBkMEdEc/N9sBq4efFVj3er+mHx0v4wO6LsYnZuLEnf859dEGbh6ME91TblE6wuj+2N0dM6g72J0bhauS3SZISIaQ0Q0p5975qp3NTo6tzL6LkbnZuO6hjAX3tUMXD1YVxgdnStA38Xo3Exc9wyy+RONK+kE/s/1YqDnfsXH+/Hy5tbXWxwdnZuG/BxlOjo3EzcsTXnhZI+uhGPuAAHctVPWjlzC7o3Hb5RoOjqlEj1Hmc7NSKk48a96fDaGiGgE0Oi9DgVlZHXOEDZ33I0WQUdHR+eyEGeyll/DQYVIAo5c84F1/qvESClDb7QQJYWuDzqXwX9OF0pkktHR0dHR0YFSYi7T0dHR0flvok8yOjo6Ojolhj7J6Ojo6OiUGLfkJCOEeFEIsUMIsVUIsVkI0egaj99KCLGwuPevwfO6CyFqFHr9hxCiwbV+js5/D10XdEqaG3ZO5kYhhGgCdAXqSyntQogQwHSDxbpaugMLgZ03WhCdmwddF3SuB7fiTiYSSJZS2gGklMlSyhMAQojbhRArhRAbhBBLhBCRnvt/CCHeFUL8LYTYLoSI89yP89zb5Pm3anGFEEJ4CyFmCiHWefrf7bn/oBBivhBisRBinxDizUJ9Bgsh9nrk+UwI8T8hRFOgGzDFsxKN9TS/Vwjxr6d982vxxun859B1QafkkVLeUj+AD7AZ2At8CLT03DcCfwOhntd9gJme6z+AzzzXLYDtnms/wOC5bgfM81y3Ahae59kF94FJQD/PdYBHHm/gQeAg4A9YcJ+viAbKAIeBII+sq4D/efrPAu4p9Jw/gLc9152BZTf6fdd/St+Prgv6z/X4ueXMZVLKLCHE7UBzoDXwrRDiOWA9UAv4TQgBoAInC3Wd4+n/pxDCTwgRAPgCXwghKgMS9xe+uNwJdBNCjPa8tgDlPNfLpZTpAEKInUAMEAKslFKmeO5/D1S5yPjzPf9uAMpfhlw6twi6LuhcD265SQZASunCvcL5QwixDRiI+wu4Q0rZ5ELdzvP6VWCFlLKHEKK8Z8ziIoBeUso9RW66Ha/2QrdcuD+ny01alT9Gfn8dnXPQdUGnpLnlfDJCiKqe1VY+dXFvw/cAoR5nKEIIoxCiZqF2fTz3mwHpntWVPxDv+f2DlynKEmCE8CwVhRD1LtH+X6ClECJQCGEAehX6XSbulaSOTrHRdUHnenDLTTK47dBfCCF2CiG2AjWA8VLKPOAeYLIQYgtuW3XTQv1ShRB/Ax8D+ZXX3gReF0L8hdukcDm8ituksFUIsd3z+oJIKeNx267XAstwR8+ke349F3jG4zSNvcAQOjpno+uCTomj5y4rBkKIP4DRUsr1N1gOH48d3QAswO2MXXAjZdK5tdB1QedyuRV3Mjcz44UQm4HtwCHgxxssj47OjULXhZsEfSejo6Ojo1Ni6DsZHR0dHZ0SQ59kdHR0dHRKDH2S0dHR0dEpMfRJRkdHR0enxNAnGR0dHR2dEuP/ATFZuFwI2plhAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn import svm, datasets\n",
    "%matplotlib inline\n",
    "\n",
    "def make_meshgrid(x, y, h=.02):\n",
    "    \"\"\"\n",
    "    根据特征向量，和网格间距参数，绘制网格点阵图\n",
    "    \"\"\"\n",
    "    x_min, x_max = x.min() - 1, x.max() + 1\n",
    "    y_min, y_max = y.min() - 1, y.max() + 1\n",
    "    xx, yy = np.meshgrid(np.arange(x_min, x_max, h),\n",
    "                         np.arange(y_min, y_max, h))\n",
    "    return xx, yy\n",
    "\n",
    "\n",
    "def plot_contours(ax, clf, xx, yy, **params):\n",
    "    \"\"\"绘制决策边界.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    ax: 绘图对象\n",
    "    clf: 分类模型\n",
    "    xx: 特征数据点\n",
    "    yy: 特征数据点\n",
    "    params: 字典参数\n",
    "    \"\"\"\n",
    "    Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])\n",
    "    Z = Z.reshape(xx.shape)\n",
    "    out = ax.contourf(xx, yy, Z, **params)\n",
    "    return out\n",
    "\n",
    "\n",
    "# 加载数据\n",
    "iris = datasets.load_iris()\n",
    "# 选取样本数据的两个特征\n",
    "X = iris.data[:, :2]\n",
    "y = iris.target\n",
    "\n",
    "C = 1.0 \n",
    "models = (svm.SVC(kernel='linear', C=C),\n",
    "          svm.LinearSVC(C=C),\n",
    "          svm.SVC(kernel='rbf', gamma=0.7, C=C),\n",
    "          svm.SVC(kernel='poly', degree=3, C=C))\n",
    "models = (clf.fit(X, y) for clf in models)\n",
    "\n",
    "titles = ('SVC with linear kernel',\n",
    "          'LinearSVC (linear kernel)',\n",
    "          'SVC with RBF kernel',\n",
    "          'SVC with polynomial (degree 3) kernel')\n",
    "plt.figure(figsize=(11,11))\n",
    "fig, sub = plt.subplots(2, 2)\n",
    "plt.subplots_adjust(wspace=0.4, hspace=0.4)\n",
    "\n",
    "\n",
    "X0, X1 = X[:, 0], X[:, 1]\n",
    "xx, yy = make_meshgrid(X0, X1)\n",
    "\n",
    "for clf, title, ax in zip(models, titles, sub.flatten()):\n",
    "    plot_contours(ax, clf, xx, yy,\n",
    "                  cmap=plt.cm.coolwarm, alpha=0.8)\n",
    "    ax.scatter(X0, X1, c=y, cmap=plt.cm.coolwarm, s=20, edgecolors='k')\n",
    "    ax.set_xlim(xx.min(), xx.max())\n",
    "    ax.set_ylim(yy.min(), yy.max())\n",
    "    ax.set_xlabel('Sepal length')\n",
    "    ax.set_ylabel('Sepal width')\n",
    "    ax.set_xticks(())\n",
    "    ax.set_yticks(())\n",
    "    ax.set_title(title)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 参数寻优"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The best parameters are {'C': 1, 'gamma': 0.1} with a score of 0.82\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "d:\\env\\pythonve\\test36\\lib\\site-packages\\sklearn\\model_selection\\_search.py:841: DeprecationWarning: The default of the `iid` parameter will change from True to False in version 0.22 and will be removed in 0.24. This will change numeric results when test-set sizes are unequal.\n",
      "  DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "grid = GridSearchCV(svm.SVC(), param_grid={\"C\":[0.1, 1, 10], \"gamma\": [1, 0.1, 0.01]}, cv=4)\n",
    "grid.fit(X, y)\n",
    "print(\"The best parameters are %s with a score of %0.2f\"\n",
    "      % (grid.best_params_, grid.best_score_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
